Fantastic Programming Project Ideas and Where to Find Them (a beginner-friendly version)

Fantastic Programming Project Ideas and Where to Find Them (a beginner-friendly version)

20 cool project ideas for Data Science, Machine Learning app development and web development and 4 ‘goldmines’ of such project ideas

Nityesh Agarwal
Nov 2 · 12 min read
We all know that working on personal projects is a really important part of learning. Its honing the discipline. Moving from theory to practice. A way of learning by doing.
But isn’t the task of getting good project ideas the one that also prevents you from building stuff, in the first place? Isn’t that a narrow bottleneck?
Aren’t you tired of those lists of programming projects ideas that suggest you to analyse Titanic dataset or the flowers dataset or build things like a To-Do list app, a snake game, a calculator, an ecommerce website or something else that no one is going to find cool?
I find them really boring because no one is ever going to be excited about using the final result. To be honest, not even me.
I believe that you can train your mind to get better ideas — anyone can think of good ideas. If you look at enough such projects and maybe work on a few, your mind will learn to recognise cool things that will be interesting to work on.
So, here I present some project ideas that I find really cool, grouped by their sources — my “goldmines” of fantastic project ideas.

Source #1: Kaggle

I believe that if you want to get into Data Science/ML, Kaggle is your one-stop shop to learn and practice the craft:

So, how would you like to work on a project that analyses:

This dataset contains the daily ranking of the 200 most listened songs in 53 countries from 2017 and 2018 by Spotify users. It contains more than 2 million rows, which comprises 6629 artists, 18598 songs for a total count of one hundred five billion streams.
And find answers to:
2: Young people survey dataset that explores the preferences, interests, habits, opinions, and fears of young people.
1010 students were asked questions regarding their:
I find the Dark net simply fascinating.
This is a data parse of marketplace data ripped from Agora (a dark/deep web) marketplace from the years 2014 to 2015. It contains drugs, weapons, books, services, and more.
Here’s some inspiration:
“This data set was made from an html rip made by reddit user “usheep” who threatened to expose all the vendors on Agora to the police if they did not meet his demands (sending him a small monetary amount ~few hundred dollars in exchange for him not leaking their info). Most information about what happened to “usheep” and his threats is nonexistent. He posted the html rip and was never heard from again. Agora shut down a few months after. It is unknown if this was related to “usheep” or not, but the raw html data remained.”
4: News headlines of India that contains 18 years of headlines focussing on India.
It contains approximately 2.9 million events published by Times of India from 2001 to 2018.
You could use this to -
You could use this meaty survey to arrive at data-backed answers to the following questions:

Source #2: Data is Plural

This is yet another source for Data Science or Machine Learning projects. It is a free email newsletter where the author sends you bunch of curious datasets each week.
Why you should analyse curious datasets for your personal projects:
  1. They are thrilling to work on — you are curious about knowing the results of the analysis yourself
  2. They are an easy way to create interesting projects — even a simple analysis on a dataset that is inherently interesting, will be interesting.
Alright, so here are some cool ones from Data is Plural’s archives:
TED talks have become an integral part of our culture.
A group of teenagers cluster near their lockers, enjoying quick conversations between classes. One of them goes a little too long and, realizing it, addresses the group and the situation by announcing, “Well, thanks for coming to my TED talk.” The rest laugh, nod their heads, and the conversational flow returns to normal before the bell sounds announcing that classes are about to begin. (From field notes by one of the authors.)
Analyse these transcripts to reveal some intricasies about our culture.
7: How couples meet and stay together. It is a survey of 4,002 adults, 3,009 of those had a spouse or main romantic partner. It even has follow-up surveys that were implemented one and two years after the main survey, to study couple dissolution rates.
An analysis can reveal answers to the following questions:
8: Electricity in rural India. The Smart Power India and the Initiative for Sustainable Energy Policy published a survey dataset that “covers 10,000 households and 2,000 rural enterprises across 200 villages in Bihar, Uttar Pradesh, Odisha, and Rajasthan.” Respondents were asked, among other things, how many hours per day they get electricity, whether they have solar panels, and the price they pay for kerosene.
Do an analysis to understand exactly how dire is the state of rural India and compare them with your own conditions.
9: Deaths on jobs Since 1992, the US Bureau of Labor Statistics’ has collected data on work-related deaths through its Census of Fatal Occupational Injuries.
You could do a detailed study of the jobs to avoid, maybe?
10: A dataset of sarcasms in TV shows like Friends and The Big Bang Theory! MUStARD is a corpus of 690 text and video clips “for research in automated sarcasm discovery.” The dataset’s 690 examples — half involving sarcasm, half not — come from Friends, The Golden Girls, The Big Bang Theory, and Sarcasmaholics Anonymous.
I bet there are lots of interesting things we could do with this hilarious dataset!

Source #1. Devpost.com

Browsing through other people’s hackathon projects on is a great way to come across good project ideas because -

How cool would it be if you created: -

12. A browser extension that referred you to a story with an opposite political view as the one that you are reading. (This will combat the effects of newsfeeds that allow people to only see posts on social media and news sites that agree with their point of view)
And tricked yourself to learn — Basic web development (HTML/CSS, Javascript, jQuery), maybe some Machine Learning

Source #4: Y. O. U.

Oh yes, I did that!
I wrote in the beginning — you can train your mind to come up with good ideas yourself.
I think Paul Graham’s advice on how to find startup ideas also kinda applies to how to find your pet project ideas.
The way to get startup ideas is not to try to think of startup ideas. It’s to look for problems, preferably problems you have yourself.
At the same time, (and this may sound like I’m contradicting myself) you don’t want to set the bar too high. You may have watched the movie The Social Network too much and hope to make the next Google or Facebook out of this project. But you shouldn’t. This will only slow down the learning, make you create unrealistic goals, and most dangerously, make you procrastinate.
Remember, your goal is not to write a billion dollar software. It is to create a program that is going to provide a stage for you to work on and simply learn from. Like, for instance:
16: While chatting with my friend, we discussed how cool it would be to build a tool to analyze our Whatsapp chats and reveal things like — #messages sent, #words sent, average no. of words per message, most common words, longest double texting streak, chat hour pattern, most shared website links and more. We later found out that we had rediscovered an idea that was really on popular on Reddit once
What’s awesome was that in the process of building it, she tuned her Python skills up a notch. Now, we might even try our hands on web development and build a website that allows anyone to run an analysis on their own chat file!
17: I use Chrome bookmarks a lot. I really need to add comments to my bookmarks so that I could save my motivation for bookmarking that awesome link. But Chrome browser doesn’t have an option to comment. That is why I built a simple Chrome extension to help me add comments to my bookmarks.
And I tricked myself to learn- Javascript, jQuery, HTML
18: When Game of Thrones released its last season a few months ago, I thought of building a script to analyze the sentiment of tweets of various Game of Thrones seasons to learn just how bad the last season was (😜)
Do this and you can trick yourself to learn — Python, Machine Learning, NLP
19: And since your goal is to learn, you shouldn’t feel bad about reimplementing some existing idea. One day I came across this popular post on Hacker News called “I taught my little brother JS and he built this videogame in a week”. I checked out the game and it was kind of addictive but really simple. I told my above mentioned friend about it and we are building a Python version of this cool game using PyGame.
20: A simple app that reminds you to follow up with important, busy people that you want to connect with. I recently read an article by Alexey Guzey on how you shouldn’t expect busy people to reply to your first message and how it is your responsibility to follow up with them. But when you have a bunch of important people to talk to, it can be a little difficult to keep track of the follow-ups. This app will do it for you and also remind you about future follow-ups.
So here’s 3 final pointers on how to come up with (sort of) cool ideas —
Do you have some problems like these yourself? It would be awesome if you tell the community about your weird, potentially cool project ideas in the comments below. I’ll start with 2 of my own weird ideas. Check them out in the comments below! And let’s make the comments of this post another goldmine of fantastic project ideas, huh?!
This is definitely not an exhaustive list of sources of cool project ideas. There are a lot more goldmines like this out there but, of course, they are difficult to find. I’ll update this post as I discover more of them.
I’ll announce any updates to this article on my Twitter, on Build To Learn newsletter and in Build To Learn Slack group.
Next in this series, I take apart the above projects one by one and give you a detailed roadmap of building and learning on the way. I start with the Whatsapp Chat Analyser project.
So, follow and subscribe to keep in touch.
I would love it if you could tell me which project are you most excited about. I’ll move it up my priority list to write its detailed roadmap.
Let me know in the comments below. You can also reach out me on both Twitter and LinkedIn.

Towards Data Science

Sharing concepts, ideas, and codes.

Comments

Post a Comment

Popular posts from this blog

Flutter for Single-Page Scrollable Websites with Navigator 2.0

Image
  Single page scrollable websites are everywhere. In this website design pattern, all the content is presented in one long-scrolling layout that contains multiple sections. Visitors can scroll or jump to a section by clicking buttons of a sticky menu. This pattern is a good fit for small content such as brochure websites, software library documentation, portfolios, and landing pages that are used to convert users [ 1 ]. Designers also love this pattern because it is simple, clean, and enables cool scroll animations. Jetbrains Mono single-page scrollable website Hive docs multi-page scrollable website In this series of articles, we will explore how to build a  single-page scrollable website  using Flutter. We will benefit from the Navigator 2.0 API to provide a good navigation experience to the users. You can find the source code of this series’ sample apps on my  Github page . The implementation details of the sample apps will be explained in the following articles: Part 2: Scroll to P
Read more

A Data Science Portfolio is More Valuable than a Resume

Image
A Data Science Portfolio is More Valuable than a Resume Benjamin Obi Tayo Ph.D. Following Nov 20  · 5 min read D ata Science is a practical field. Hands-on skills are very important especially when you are interested in working outside academia as a practicing data scientist. In academia, you need more theoretical and research skills. While in-depth knowledge in the theoretical foundations of data science is important, as a practicing data scientist, hands-on experience is very crucial and one way to showcase you hands-on skills is via building a very great portfolio. Companies interested in hiring you would definitely be asking you for a portfolio, as it gives evidence of your strengths in fundamental data science concepts. While a data science resume is important, a data science portfolio is more valuable than a resume. This article will discuss 3 important platforms that can be used for portfolio building. Before d
Read more

Better File Storage in Oracle Cloud

Image
Use Oracle Application Express with Oracle Cloud Infrastructure REST APIs to upload, download, and manage big files in Oracle Cloud. By Adrian Png November 27, 2019 Oracle Cloud Infrastructure  provides a comprehensive suite of REST APIs for accessing and managing cloud resources. For example, administrators can use REST APIs to create and manage autonomous databases and initiate backups. Oracle Application Express (Oracle APEX)  makes working with REST easy. The  APEX_WEB_SERVICE  package, available since Oracle APEX 4.0, provides developers with a function for consuming REST APIs. Oracle APEX 5.0 introduced the  APEX_JSON  package, allowing easy parsing of JSON responses. Oracle APEX 18.1 introduced the Web Source Module (WSM), which enables developers to access REST services and use the data in APEX components such as reports, interactive reports, and interactive grids. These features, when used together with Oracle Cloud Infrastructure REST APIs, enable APEX developers
Read more