Queen of Denmark - New Years Speeches: Web scraping and NLP-project

A visualization project by Alexander Ibsen Dernild & Max Festersen Hansen.

"Topics can be hard to document correctly. By using this tool, you can find relevant topics and the words within on the topics tab. From there you can then choose to explore the sentiment, countries, or statistics to get a better understanding of the topic. You could also do your own thing and look at pretty graphs and statistics. We won't judge."

See live on: https://app.dernild.com/

|██████████|

100% complete.

Tasks

• Create interactive dashboard to interact with findings.
• Collect speeches.
• Collect english translated speeches.
• Generate files with meta info about speeches.
• Clean speech text.
• Preprocessing: Tokenization, stemming, lemmatization etc.
• Create custom stop-word list.
• Build Structural Topic Model (STM).
• Calculate Polarity scores (Sentiment Analysis).
• Implement worldmap and map placese the Queen mentions.
• Feature Charts frequency of words.
• Feature sentiment charts.
• Add filtering options for year, words and language.
• Reactive uodate to word filter by interacting with topic model, with options for integration.
• Reactive language switch for data.
• Create reative list of data sources.
• Scrape information about the Queen from wikipedia.
• Reactive word additions to featured words from topic model
• Use CRON script to automize datacollection.
• Write 4 page report.
• Create short video demonstrating the tool.

[ ] = To do

x = Done

* = Stuck untill another post can progress

[R] = Ready to review

Task... Maybe

• Translate dashboard to Danish, and give option to use Danish/English.
• Use ai to generate speeches.
• Analyze speech lengths, sentences said and feature relevant charts.
• Optimize for mobile view
• Optimize performance (make vectorized solutions rather than for loops)