One Click Answer: Data Science

What is pre-requisites for data Science?

April 26, 2020
By Sanjeev Dubey
0 Comments

Data Scientist, Big Data, Artificial Intelligence, Machine Learning, Python, R, Predictive Modelling, Statistical Analysis, Business Intelligence, Deep Learning, Business Analytics

The prerequisites required in order to learn data science now. Many of the subscribers have been asking this particular question even I'm getting messages on LinkedIn saying that I'm from commerce background can I move it to data science domain, what are the prerequisite crèche, I'm a mechanical engineering student can I move into data science and many more kind of questions are actually coming up.

Always remember one thing guys data science is a technique which can be applied in any domain if you have a specific use case where machine learning or deep learning fits definitely go ahead and you can apply in that particular domain. So I'm going to mention some of the prerequisites that you need to have so that you can easily make a transition towards data science. I'm going to mention four prerequisites:

i) the first prerequisite is that you need to have some basic programming knowledge now when I say to have some basic programming knowledge like C and C++. If you have done some programming initially, making a transition towards data science can be very very easy because of some of the programming languages that you will be used in data science such as Python is a very easy programming language to learn,

ii) the second prerequisite is that you need to be at least good at maths, you should also have some interest in maths if you don't have any interest and if you're not good at maths guys, I think it'll be very very difficult because if I talk about data science if I talk about every modules in a lifecycle of a data science project like feature engineering, feature selection, model creation everywhere maths is actually involved.

iii) The third prerequisite that I'm actually specifying is which is called statistics, again guys math and statistics both are important. Now if I just talk about statistics I am basically talking about data and are taking up the data from any use case you are doing a lot of analysis here. You're trying to work on the population. You are trying to work on the sample of data. In short, it is all interrelated with statistics so all the mathematical equations that you will be applying will be basically applying on that specific data. Both Math and Statistics are very much important, this is a very important prerequisite, and I know most of the colleges that you study about statistics is all about Mean, Median, and Mode. It is not just that guys if you just try to consider a very good use case you'll be able to learn a whole lot of things and again if you are not good at math side I would suggest that you go and follow a Khan Academy of statistics they're a whole lot of videos different use cases that I actually discussed over there and it is explained pretty much better

iv) the next prerequisite is that you need to have some basic knowledge of databases. Database plays a very important role and in a data science project because all the data initially stored in some databases and then you pick up that particular data, then you do all the feature engineering process, feature selection process and you do basically you finally create a model and do the deployment. Some basic knowledge of database is also very important because at the end of the day, data will be stored somewhere someplace and you have to retrieve that particular data and do all this particular process. It is very important that you need to have the knowledge of databases like SQL MongoDB, No SQL, SQL database and some other type of types of databases. So if you have that particular knowledge and if you are also a big data engineer who has actually worked in Hadoop database architecture and all definitely if you move towards data science you will basically be called as a full-stack data scientist because you know the big data part also you know the data science part also that basically means you will be able to do all the related work with respect to the data science.

So, yes these are the pretty much important prerequisites that basically required, and this prerequisite, if you have definitely your transition towards data science, will be smoother. If you don't have then you have to put some more effort while you're learning those things.

Data Science

How Data Scientist work?

April 26, 2020
By Sanjeev Dubey
0 Comments

Are you one of the many who dreams of becoming a data scientist? If you are passionate about data science, I will tell you how does it really work. Being a data scientist lets us see how a day inner life goes while working on a data science project.

Well it is very important to understand the business problem first in meeting with the clients. I ask relevant questions to understand and defines objectives for the problem that needs to be tackled. Being a purist soul who asks a lot of questions is one of the many traits of a good data scientist. Now start cares up for data acquisition to gather and scrape data from multiple sources like SAP servers logs databases API and online repositories. Oh, it seems like finding the right data takes both time and effort. After the data is gathered comes data preparation this step involves data cleaning and data transformation. Data cleaning is the most time-consuming process as it involves handling many complex scenarios here. I usually deal with inconsistent data types, misspelled attributes, missing values, duplicate values, and whatnot.

Then to increase the transformation I usually modify the data based on defined mapping rules in a project. ETL tools like talent and Informatica are used to perform complex transformations that help the team to understand the data structure better than understanding what you actually can do with your data is very crucial. For that, I usually do exploratory data analysis with the help of EDA able to define and refines the selection of feature variables that will be used in moral development. However, if I escape this step, I might end up choosing the wrong variables, which will produce an inaccurate model.

Thus exploratory data analysis becomes the most important step, now I proceed to the core activity of a data science project such as data modeling. I repetitively apply various types of machine learning techniques like KNL, decision tree, knife phase to the data to identify the moral that best fits the business requirement. I usually train the models on the training data set and tests them to select the best performing model. Python can be used for modeling the data however; it can also be done using R and SAS well the trickiest part is not yet over. Visualization and communication must be performed to understand the data effectively.

Now, meets the clients again to communicate the business findings in a simple and effective manner to convince the stakeholders that we use tools like tableau, power bi, and QlikView that can help in creating powerful reports and dashboards. Then finally deploys and maintains the model that tests the selected model in a pre-production environment before deploying it in the production environment, which is the best practice. Right after successfully deploying it we uses reports and dashboards to get real-time analytics further. There is also a need to monitors and maintains the performance of the project well. That is how I complete the data science project. We have seen the daily routine of a data scientist is Koller fun has many interesting aspects and comes with its own share of challenges.

Now let us see how data science is changing the world. Data science techniques along with genomic data provide a deeper understanding of genetic issues in reaction to particular drugs and diseases logistic companies like DHL FedEx have discovered the best routes to ship the best suited time to deliver the best mode of transport to choose, thus leading to cost efficiency with data science. It is possible to not only predict employee attrition but to also understand key variables that influence employee turnover. In addition, the airline companies can now easily predict flight and notify the passengers beforehand to enhance their travel experience well. If you're wondering there are various rules offered to a data scientist like data analyst, machine learning engineer, deep learning, data engineer, and of a course data scientist.

The median base salaries of the data scientists can range from 95 thousand dollars to 165 thousand dollars so that was about the data science are you ready to be a data scientist if yes then start today the world of data needs you. That is all from my side, thank you for reading the article, comment below the next topic that you want to learn.

Data Science

What Really is Data Science?

April 26, 2020
By Sanjeev Dubey
0 Comments

Data science is not about making complicated models. It's not about making awesome visualizations. It's not about writing code. Data science is about using data to create as much impact as possible for your company. Now, the impact can be in the form of multiple things. It could be in the form of insights in the form of data products or in the form of product recommendations for a company. Now to do those things, then you need tools like making complicated models or data visualizations or writing code. But essentially as a data scientist, your job is to solve real company problems using data and what kind of tools you use we don't care about.

Now there's a lot of misconception about data science, especially on YouTube and I think the reason for this is because there's a huge misalignment between what's popular to talk about and what's needed in the industry. So because of that, I want to make things clear.

I am a data scientist working for a GAFA company and those companies really emphasize on using data to improve their products. Therefore, this is my take on what is data science. Before data science, we popularized the term data mining in an article called from data mining to knowledge discovery in databases in 1996 in which it referred to the overall process of discovering useful information from data. In 2001, William S. Cleveland wanted to bring data mining to another level. He did that by combining computer science with data mining. Basically, he made statistics a lot more technical which he believed would expand the possibilities of data mining and produce a powerful force for innovation.

Now you can take advantage of computing power for statistics and he called this combo data science. Around this time, this is also when web 2.0 emerged where websites are no longer just a digital pamphlet, but a medium for a shared experience amongst millions and millions of users. These are web sites like MySpace in 2003 Facebook in 2004 and YouTube in 2005. We can now interact with these web sites meaning we can contribute post comments like upload share leaving our footprint in the digital landscape we call the Internet and help create and shape the ecosystem we now know and love today. In addition, guess what? That's a lot of data so much data, it became too much to handle using traditional technologies. Therefore, we call this Big Data.

That opened a world of possibilities in finding insights using data. However, it also meant that the simplest questions require sophisticated data infrastructure just to support the handling of the data. We needed parallel computing technology like MapReduce, Hadoop, and Spark so the rise of big data in 2010 sparked the rise of data science to support the needs of the businesses to draw insights from their massive unstructured data sets. So then, the journal of data science described data science as almost everything that has something to do with data Collecting, analyzing, and modeling. Yet the most important part is its applications. All sorts of applications.

Yes, all sorts of applications like machine learning. So in 2010 with the new abundance of data, it made it possible to train machines with a data-driven approach rather than a knowledge-driven approach. All the theoretical papers about recurring neural networks, support vector machines became feasible. Something that can change the way we live and how we experience things in the world. Deep learning is no longer an academic concept in this thesis paper. It became a tangible useful class of machine learning that would affect our everyday lives. So machine learning and AI dominated the media overshadowing every other aspect of data science like exploratory analysis, experimentation, ... And skills we traditionally called business intelligence.

So now the general public thinks of data science as researchers focused on machine learning and AI but the industry is hiring data scientists as analysts. So there's a misalignment there. The reason for the misalignment is that yes, most of these data scientists can probably work on more technical problems but big companies like Google, Facebook, Netflix have so many low-hanging fruits to improve their products that they don't require any advanced machine learning or the statistical knowledge to find these impacts in their analysis. Being a good data scientist isn't about how advanced your models are. It's about how much impact you can have with your work. You're not a data cruncher. You're a problem solver. You're strategists. Companies will give you the most ambiguous and hard problems.

Now the thing that's less known is the stuff in between which is right here everything. Surprisingly this is actually one of the most important things for companies because you are trying to tell the company what to do with your product. Therefore, what do I mean by that? Therefore, I am analytics that tells you using the data what kind of insights, what are happening to my users and then metrics. This is important because what is going on with my product? You know, these metrics will tell you if you are successful or not. Then you know to be testing, of course, Experimentation that allows you to know, which product versions are the best. Therefore, these things are actually important but they are not so covered in media. What's covered in media is this part. AI, deep learning. We have heard it on and on about it, you know.

But when you think about it for a company, for the industry. It's actually not the highest priority or at least it's not the thing that yields the most result for the lowest amount of effort. That's why AI and deep learning is on top of the hierarchy of needs and these things may be testing analytics. They're actually way more important for the industry so that's why we're hiring a lot of data scientists that do that. So what do data scientists actually do?

Well that depends on the company because of its size. So for a start-up you kind of lack of resources. So you can only kind of have one Data Scientists. So that one data scientist he has to do everything. So you might be seeing all this being data scientists. Maybe you won't be doing Artificial Intelligence or Deep Learning because that's not a priority right now. But you might be doing all of these. You have to set up the whole data infrastructure. You might even have to write some software code to add logging and then you have to do the analytics yourself, then you have to build the metrics yourself, and you have to do A/B testing yourself.

That's why for startups if they need a data scientist this whole thing is data science, so that means you have to do everything. But let's look at medium-sized companies. Now, finally, they have a lot more resources. They can separate data engineers and data scientists. So usually in the collection, this is probably software engineering. And then here, you're going to have data engineers doing this. Depending on if you're medium-sized company does a lot of recommendation models or stuff that requires Artificial Intelligence, then Data Scientists will do all these. So as a Data Scientist, you have to be a lot more technical. That's why they only hire people with PhDs or masters because they want you to be able to do more complicated things.

So let's talk about a large company now. Because you're getting a lot bigger you probably have a lot more money and then you can spend it more on employees. So you can have a lot of different employees working on different things. That way the employee does not need to think about this stuff that they don't want to do and they could focus on the things that they're best at. For example, me and my untitled large company I would be in analytics so I could just focus my work on analytics and metrics and stuff like that. So I don't need to worry about Data Engineering or Artificial Intelligence, deep learning stuff. So here's how it looks for a large company Instrumental logging sensors. This is all handled by software engineers Right? And then here, cleaning and building data pipelines. This is for data engineers. Now here, between these two things, we have Data Science Analytics. That's what it's called.

But then once we go to AI and deep learning, this is where we have research scientists or we call it data science core, and they are backed by and now engineers which are machine learning engineers. Anyways, so in summary, as you can see, data science can be all of this and it depends on what company you are in and the definition will vary. So please let me know what you would like to learn more about AI deep learning, or A/B testing, experimentation. Depending on what you want to learn about leave a comment down below so I could talk about it or I could find someone who knows about this and I can share the insights with you.

Hope you have a wonderful day. Hope this was helpful. Thanks for reading.

Pages

What is pre-requisites for data Science?

How Data Scientist work?

What Really is Data Science?

About me

Popular Posts

Find us on Facebook

Pinterest

Labels

recent posts

Blog Archive

Instagram

Follow me

Socialize!