Subrat's Technical Blog

Overview It is said that Data Scientist spends 80% of their time in preprocessing the data, so lets deep dive into the data preprocessing pipeline also known as ETL pipeline and let's find out which stage takes the most time. In this blog post, we will learn how to extract data from different data sources. Let's take a real-life dataset so it’s easier to follow. This lesson uses data from the World Bank. The data comes from two sources: World Bank Indicator Data  — This data contains socio-economic indicators for countries around the world. A few example indicators include population, arable land, and central government debt. World Bank Project Data  — This data set contains information about World Bank project lending since 1947. Types of Data files CSV  — CSV stands for comma-separated value. This is how the file looks id,regionname,countryname,prodline,lendinginstr P162228,Other,World;World,RE,Investment Project Financing P163962,Africa,Democratic Repub...

In this surrealistic time of the COVID 19 virus, a lot of conferences where I was going to speak this year were cancelled, so I decided to write my presentation in the form of an article. Not sure if I’m able to tell this story somewhere in the world this year, so here it is! This blog is also available on  Oracle.com .  Delivering high quality software in a 24*7 environment is a challenge for a lot of teams these days. Teams of multinationals need to deliver more functionality more quickly. Some of these teams are able to deliver new software every hour. How is it possible to meet these demands? DevOps You must have heard the term many times these days. Maybe you already work like this in your team. Anyway, methods of working are changing rapidly, and the DevOps’s way of working is the way a lot of teams aim towards. Either to start with or to improve their own way of working and meet the high demands of their clients, business teams or goals set by their company. ...