Lab: Analyzing Trump Tweets

Download: lab_tweets.py

This lab is different from the ones before it. There is no starter repository to fork and no doctests to turn green. You’ll write “fully open-ended” code, the kind where nobody hands you a scaffold and tells you the moment you’re done. That is the point: real problems arrive without a test suite, and this is practice for meeting one.

You’ll analyze every tweet Donald Trump sent from 2009 to 2018, and get practice loading datasets stored in JSON files, counting patterns in text, and making a plot in Python. The same dataset powers two well-known projects: an analysis of which tweets Trump wrote himself versus which his staffers sent, and a search engine for all of his tweets.

Because nothing here is auto-graded, your grade comes from what you produce: a short README.md showing a formatted table and a bar chart, plus the code that made them. Everything below is also written into the top of lab_tweets.py itself, so you can work from either place.

Part 0: set up the project

You’ll upload files to GitHub to submit this lab, so start by making a home for them.

  1. Create a new GitHub repository through the GitHub website.
  2. Clone that repository onto your computer.
  3. Download lab_tweets.py and copy it into your project folder.

Part 1: download the data

The repository https://github.com/bpb27/trump_tweet_data_archive holds an archive of Trump’s tweets.

  1. Download the files named master_*.json.zip, where * is a year. There should be 10 of them, one for each year from 2009 to 2018.
  2. Unzip them into the project folder you made in Part 0. You’ll get a set of files named like master_2009.json.

This particular archive stops in 2018 because its maintainer moved to a newer archive that runs through the present (and includes messages sent on Truth Social). That newer data is a little more work to get at, so we use the older archive here. Using the latest data instead is worth extra credit; see the last section.

Part 2: analyze the data

Modify lab_tweets.py so that it:

  1. Opens each JSON file and loads it with the json library. Each file holds a list of tweets, and if you concatenate every file’s list together you get one list of every tweet Donald Trump ever sent.

  2. Prints the total number of tweets.

  3. Counts how many tweets contain each of these keywords: Obama, Trump, Mexico, Russia, and Fake News.

    Each keyword can appear with many different capitalizations, and your program must count it no matter how it’s written: OBAMA, obama, and ObAmA all count as an occurrence of Obama. The easiest way to handle this is to lower-case the tweet text and use the in keyword, exactly as in the reading.

  4. Prints the count for each of these words.

A correct program produces these numbers, so you can check yourself against them:

len(tweets)= 36307
counts= {'trump': 13924, 'obama': 2712 ... }

Once those match, take it further:

  1. Choose at least 3 more interesting words or phrases of your own to count, and modify your program to display them.
  2. Calculate the percentage of tweets that contain each word (both your new words and the original five).
  3. Display the results in a Markdown table, with every word right-justified and every percent printed to two figures on each side of the decimal, as shown here:
| phrase            | percent of tweets |
| ----------------- | ----------------- |
|              daca | 00.17             |
|         fake news | 00.92             |
|  mainstream media | 00.06             |
|            mexico | 00.55             |
|             obama | 07.47             |
|            russia | 01.13             |
|             trump | 38.35             |
|              wall | 00.91             |

There are many ways to line text up like that in Python, but the cleanest is f-string formatting. Here is a short cheat sheet, written as doctests you can run:

>>> name = "Alice"
>>> f"{name:<10}"
'Alice     '
>>> f"{name:>10}"
'     Alice'
>>> f"{name:^10}"
'  Alice   '
>>> f"{name:*^10}"
'**Alice***'

>>> pi = 3.14159265
>>> f"{pi:.2f}"
'3.14'
>>> f"{pi:.4f}"
'3.1416'
>>> f"{pi:8.2f}"
'    3.14'
>>> f"{pi:08.2f}"
'00003.14'

  1. Plot the results in a bar graph.

    We haven’t covered plotting in class, on purpose: figuring out a new library from its documentation is a skill this lab is built to exercise. The usual choice is matplotlib, and this w3schools tutorial is a fine place to start. You’re also welcome to ask your favorite AI.

Submission

Write a properly formatted README.md that contains:

  1. the Markdown table your program produced,
  2. the image your program produced, and
  3. a short description (one sentence is fine) of the table and image above.

Make sure your repository also holds:

  1. your modified lab_tweets.py, and
  2. the image file your program saved.

There are no GitHub Actions to pass for this lab; it isn’t auto-graded. When your repository has the code, the image, and the README.md, submit its URL on Gradescope.

Extra credit

Two opportunities, worth one point each:

  1. Use the latest data. If you use the current dataset instead of the files in the 2009–2018 archive, you earn +1. Instructions for getting it are under “Can I have the data?” in the Trump Archive FAQ.
  2. Find something else in the data. Make a plot of something interesting that does not use the text key: what time of day does Trump tweet most often? what state does he tweet from most often? Add it to your repository for +1.