Sunday, 11 December 2016

Automatic Hashtag Maker for Newsbot

Newsbot is a twitter-bot which tweets news headlines from various verified news sources. To increase its reach and engagement I had created a rudimentary version of an auto-hashtag maker. The algorithm is very basic. It just crawls the URL of the article and filters out the stop words (Eg: to,an,a,the..). It then analyses and sorts the words based on their density. It then compares these words with the once already in the headlines and if there is any match, it uses the matched word as the hashtag. Nothing fancy. It did its job of increasing the views per tweet and added some followers too but most of the hashtags were totally hilarious.

As you can see above, the hashtags were not at all informative. But sometimes this method did give an average hashtag...
And sometimes pretty good ones too...
As this algorithm does not take into account the importance of words in shaping the meaning of a sentence, it doesn't make powerful hashtags. 

The 2nd version, which the Newsbot now uses, utilizes Brill Tagger to analyze parts of speech. The first step is isolating proper nouns and common nouns. Proper nouns are given more importance as they may point towards a personality or a place. If there are no proper nouns in the sentence, then the script manipulates the common nouns to produce meaningful hashtags. This algorithm is still under testing. You can view the tweets and share your views, I will try to make the necessary changes.  It is still a work in progress and once its complete I will share the code.