In this post, I am going to show you how to query the Twitter API in Golang and tidy up the response output. Note that this code includes no go routines or channels for concurrency and has all the code in a single file, which is less than ideal. Tomorrows post will show you how to resolve these issues in a much more manageable code base.
The first thing we do, as always, is we import the modules we require. Then we define two struct types. As I spoke about in this article, a struct is a data type in Golang, it can also be compared to a class in an object oriented programming language. A struct gives us the ability to set specific fields for a type of data.
Here, I’ve defined a struct for a tweet and a struct for a user. Each will have a number of attributes. We will be interacting with these structs later, so it should become clear why I’ve used them.
package main import "github.com/amit-lulla/twitterapi" //https://godoc.org/github.com/amit-lulla/twitterapi import "fmt" import "strings" import "time" import "log" import "net/url" import "math" type user_details struct { handle string url string timezone string StatusesCount int64 location string CreatedAt string AccountAge float64 FriendsCount int FollowersCount int FavouritesCount int } type tweet_details struct { CreatedAt string Age float64 FavoriteCount int RetweetCount int }
Next, let’s have a look at the main function. Here, I only make two function calls. The first, is a call to get user details and the second is a call to get the posts for that user.
func main() {
//query API for specific handle
user_output := searchProfile("BBCSport")
getPosts("BBCNews")
fmt.Println(user_output)
}
Next, I have a connect function, where I use my API keys to create an API session in my Golang code. We’ll be referencing this function from other functions a little later.
func connect() *twitterapi.TwitterApi { twitterapi.SetConsumerKey("x") twitterapi.SetConsumerSecret("x") api := twitterapi.NewTwitterApi("x", "x") return api }
Now we get to the interesting bit. Here is the searchProfile function. This function takes the profile name argument as a string and returns a user_type struct as the function output.
So, we make a call to the connect() function in order to make an API session. We then use that API object and make a call to the GetUsersShow function. The object returned is a searchResult struct. We can then extract specific fields from that struct (as we don’t want all of them).
The date format in the response is awful it’s in this format: Fri Apr 09 12:53:54 +0000 2020. I pass this into the CalcAge function which calculates how many days ago the account was created. We’ll look at that function a bit later.
Finally, we store our output fields into our user_details struct and return it.
func searchProfile(profileName string) user_details {
api := connect()
searchResult, _ := api.GetUsersShow(profileName, nil)
FavouritesCount := searchResult.FavouritesCount
FollowersCount := searchResult.FollowersCount
FriendsCount := searchResult.FriendsCount
CreatedAt := searchResult.CreatedAt
Location := searchResult.Location
StatusesCount := searchResult.StatusesCount
TimeZone := searchResult.TimeZone
URL := searchResult.URL
handle := searchResult.Name
acctAge := CalcAge(CreatedAt)
output := user_details{
handle : handle,
url : URL,
timezone: TimeZone,
StatusesCount: StatusesCount,
location: Location,
CreatedAt: CreatedAt,
AccountAge: acctAge,
FriendsCount: FriendsCount,
FollowersCount: FollowersCount,
FavouritesCount: FavouritesCount}
return output
}
Next, I have the getPosts function, which again takes the profileName as an input but you’ll notice that it has no return value. That’s why in the main function, I didn’t assign the output to a variable. I just call the function because I want something to happen, I don’t need to store the value.
Again, we create a new API object and make a query to it. This time we use the GetUserTimeline function. This requires a url value to be the input, so I’ve used the url.parseQuery function to be able to convert my string into an acceptable format for the function.
Again, the output is saved to a variable called searchResult. The output this time is an iterable, so we loop over it and extract the values we want. As part of this I pass the horrible CreatedDate value into a function called CalcAge, which will be a bit further down this article. This calculates how many days old the post is. That way, we can easily select a custom timeframe of tweets we’re interested in. I also pass the date into the CleanDate function which converts our horrible date format described above into YYYY-MM-DD.
In each iteration, I add the values to the tweet_details struct. In this example, I just print the values, but you can save to a database or a file or handle it however you please.
func getPosts(profileName string) {
api := connect()
v, _ := url.ParseQuery("screen_name="+profileName+"&count=100&include_rts=False&tweet_mode=extended")
searchResult, _ := api.GetUserTimeline(v)
for _, value := range searchResult {
CreatedAt := value.CreatedAt
FavoriteCount := value.FavoriteCount
RetweetCount := value.RetweetCount
Posted := CalcAge(CreatedAt)
CreatedDate := CleanDate(CreatedAt)
rounded := math.Floor(Posted*100)/100 //rounds number
output := tweet_details{
CreatedAt: CreatedDate,
Age: rounded,
FavoriteCount: FavoriteCount,
RetweetCount: RetweetCount,
}
fmt.Println(output)
}
}
Finally, let’s look at our two cleanup functions. The first, is the CleanDate function. It takes our date in the format: Fri Apr 09 12:53:54 +0000 2020 and converts it to YYYY-MM-DD through string splits and a quite ugly if statement.
func CleanDate(CreatedAt string) string {
month := strings.Split(CreatedAt, " ")[1]
day := strings.Split(CreatedAt, " ")[2]
year := strings.Split(CreatedAt, " ")[5]
if month == "Jan" {
month = "01"
} else if month == "Feb" {
month = "02"
} else if month == "Mar" {
month = "03"
} else if month == "Apr" {
month = "04"
} else if month == "May" {
month = "05"
} else if month == "Jun" {
month = "06"
} else if month == "Jul" {
month = "07"
} else if month == "Aug" {
month = "08"
} else if month == "Sep" {
month = "09"
} else if month == "Oct" {
month = "10"
} else if month == "Nov" {
month = "11"
} else if month == "Dec" {
month = "12"
}
full_date := year + "-" + month + "-" + day
return full_date
}
And finally, the calcAge function, which converts the date format just as above, then casts it as a date and subtracts the date from now() to give us an age.
func CalcAge(CreatedAt string) float64 {
month := strings.Split(CreatedAt, " ")[1]
day := strings.Split(CreatedAt, " ")[2]
year := strings.Split(CreatedAt, " ")[5]
if month == "Jan" {
month = "01"
} else if month == "Feb" {
month = "02"
} else if month == "Mar" {
month = "03"
} else if month == "Apr" {
month = "04"
} else if month == "May" {
month = "05"
} else if month == "Jun" {
month = "06"
} else if month == "Jul" {
month = "07"
} else if month == "Aug" {
month = "08"
} else if month == "Sep" {
month = "09"
} else if month == "Oct" {
month = "10"
} else if month == "Nov" {
month = "11"
} else if month == "Dec" {
month = "12"
}
full_date := year + "-" + month + "-" + day
//define parsedDate as a time data type
var parsedDate time.Time
//convert our string datatype to be of type time
parsedDate, err := time.Parse("2006-01-02", full_date)
if err != nil {
log.Fatalln(err)
}
today := time.Now()
//subtract created date from today to get age
age := today.Sub(parsedDate).Hours() / 24
return age
}
