What is Sentiment Analysis ?
Sentiment analysis (SA) in the context of Data Science is analysis of text to define the customer sentiment on a topic or product. Broadly speaking sentiment analysis extracts information on Polarity (Feels Good/ No Thankyou/ Umm ok!) & Context — Subjectivity (Influenced by a personal opinion/feeling) or Objectivity (without being influenced by a personal opinion/feeling). Definition of ‘Sentiment Analysis’ according to the Data Science website towardsdatascience.com is as below :
“Sentiment analysis is contextual mining of text which identifies and extracts subjective information in source material, and helping a business to understand the social sentiment of their brand, product or service while monitoring online conversations.” — towardsdatascience.com
”Sentiment is based on language content, semantic constructs and context” — Sailen Kotecha
Business Use Cases for SA:
- Opinion Polls before Election For eg: Finding out the sentiment of Candidate X vs Candidate Y and the chance of winning based on that pre-election and post-elections.
- Customer Sentiment anticipating launch of a new product For eg: A large phone manufacturer announcing in it’s developer conference a new Tablet is being launched in 3 months and business development wants to use social media to understand how the new product announcement is being perceived by the consumer : Are people excited about the launch or the sentiment is neutral or negative
- Product feedback, For eg: A large camera manufacturer has launched a new camera model and they want to know what the customer sentiment is on their product. If a large percentage of sentiment is negative it could signal a potential fault with the camera which wasn’t detected in Quality Control (QC) after go-to market.
- Feedback to Negative Sentiment for a Service Outage : For eg: A Big Telco is having an outage and people are complaining on Social media, using sentiment analysis any tweet/post with a negative sentiment can be used to send an automated reply to the customer.
- For Airlines it could be used to send automated reply to customers via Private message who may have issues with refunds/missed flights/baggage etc.
In this blog post we are going to do basic sentiment analysis based on polarity i.e positive, negative or neutral. Sentiment Analysis is a very wide data science topic and there is much more to explore there however it is beyond the scope of this article. The solution we are building is on Oracle Cloud Infrastructure (OCI) services, which is cost-effective and one of the simplest ways to do this. The same solution on AWS would require twice the moving parts and complexity.
Part 1 — OCI Architecture & List of Components
Part 2 — Build & Deploy the Components on OCI
Part 3 — Run the Twitter Ingestion and Create the Visualization from Oracle Analytics Cloud
Part 1 — OCI Architecture & List of Components
Components
- Twitter Developer Account
- Python 2.7 Twitter Listener running on OCI Compute VM
- Oracle Autonomous Database Instance
- Oracle Analytics Cloud Instance
Part 2 — Build & Deploy the Components on OCI
- Create a Twitter application (App Permission : Read, Write, and Direct Messages) with a developer account and create API Key, API Secret Key, Access Token and Secret.
#Create twitter api application keys
#consumer key, consumer secret, access token, access secret.
ckey=’Do not’
csecret=’Disclose’
atoken=’These’
asecret=’Secret Keys’
2. Launch a OCI VM with Python 2.7.5 installed and pip install ‘cx_Oracle’ & ‘tweepy’
$ pip install cx_Oracle
$ pip install tweepy
3. Create an Oracle Autonomous Database :https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/tutorial-getting-started-autonomous-db/index.html
4. Create a Python Script on a compute VM using Twitter & Autonomous Database credentials
The code for the Tweepy listener is based on Arthur Daytons awesome post. I have added some extra functionality; tweaked it to work on Autonomous Database, Twitter bot included to send a Tweet every-time the Twitter listener is fired-up & choose different phrases/text attributes using ‘AND’ ‘OR’ operator in Tweepy
## Twitter Streamer Python2.7 Code ##
5. Create Twitter Table and View in Autonomous Database
The JSON table feature has been available in Oracle Database since version 12.1 : https://docs.oracle.com/database/121/SQLRF/functions092.htm
6. Create Oracle Analytics Cloud (OAC) Instance : https://docs.oracle.com/en/cloud/paas/analytics-cloud/index.html
Part 3 — Run the Twitter Ingestion and Create the Visualization on Oracle Analytics Cloud
Step 1. Run the Python Tweepy Listener
$ python StreamTweetsOracleUpdateStatus.py
Tweet Bot sends Tweet before starting the Stream
Step 2. Query Autonomous Database to Verify Tweets Streaming in
Step 3. Create Connection & Dataset for the Autonomous Database on OAC
Step 4. Create OAC Dataflow and Add Sentiment Analysis on the ‘tweet’ field
https://docs.oracle.com/en/cloud/paas/analytics-cloud/acubi/add-sentiment-analysis-data-flow.html
Run the Data Flow
Step 5. Finally Create Tweet Sentiments Visual using OAC Sentiment Analysis Dataflow
Summary:
We have created a cloud-native sentiment analysis social media dashboard on OCI, which can ingest tweets at an astonishing rate. This demo was created using the smallest Autonomous Database shape of 1ocpu & 1 TB storage and OAC with the smallest shape.
References for Solution Build:
1 https://developer.oracle.com/dsl/mastering-oracle-python.html
2 http://www.vlamis.com/blog/2016/10/3/twitter-live-feed-with-oracle-database-as-a-service-and-business-intelligence-cloud-service
3 https://pythonprogramming.net/twitter-api-streaming-tweets-python-tutorial/
4 https://dinodba.github.io/dinosaurdba/tweetoracle
5 https://docs.oracle.com/en/cloud/paas/analytics-cloud/acubi/add-sentiment-analysis-data-flow.html
6 https://www.infosysblogs.com/oracle/2018/04/sentiment_analysis_with_twitte.html
7 https://docs.oracle.com/database/121/SQLRF/functions092.htm
No comments:
Post a Comment