Qodex.ai

Twitter more than 100 tweets

Number of APIs: 1

About

This collection is configured to get more than 100 tweets for a specific search and save them as a collection variable. By default, the Twitter API only returns 100 results per search. But they do allow for pagination (requesting old data) if you set up the requests properly.

By capturing the next_token if it's available in a response, it can be included in subsequent API requests to capture all the tweets. This collection automatically handles the capturing and setting of the next_token, so you can get all of the results for a particular search.

Twitter documentation on pagination available here.

Community forum question

This question has come up often - how do I get more than 100 tweets?. You can read some other queries and responses on our community forum [here]

Using this collection

  1. Fork this collection into your own workspace.
  2. Enter your Twitter bearer token in the collection variables.
  3. Change the query param in the Get all tweets request.
  4. Use the [Collection Runner] to run the request multiple times. The loop will stop once there are no more new tweets. If you manually stop the collection run, you will not have the tweets saved.
  5. Look at the collection variable called allTweets to see a stringified version of all the found tweets.

Disclaimer

Some tweet searches may return a very large number of tweets. This collection saves all of the tweets as a collection variable in Qodex. If you're doing a search that returns tens of thousands of tweets, you may experience some lag.

  1. Get all tweets GET https://api.twitter.com/2/tweets/search/recent?query=cute animals&max_results=10&tweet.fields=author_id,conversation_id,created_at,geo,id,lang,source,text&user.fields=created_at,description,entities,id,location,name,url,username