While working in the field of Artificial Intelligence, the most important thing is to be a continuous learner, The amount of research going on in this field is tremendous and it is getting upgraded every day, even as you are reading this article there might be another breakthrough in AI.
So what are we trying to achieve with AI?
To answer that question we would have to look at the state-of-the-art work that has been done till now in AI and the answer would be as clear as it would be “To Provide Machine with the capabilities such as Humans”.
Speaking of Human-like capabilities, AI has Natural Language Processing (NLP), with NLP, we are providing machines with language capabilities.
Today, we can enter our house and can control every surrounding accessory with just a call of our voice(Alexa, Siri, Google home) are some of the voice assistants that came into existence only because of NLP.
In NLP, we achieved a milestone when Google recently showcased their Duplex in which an AI agent makes calls on the user’s behalf and makes reservations/appointments for them.
I hope someday we might have our very own personal Jarvis #starkfan
By now, you must have gotten an idea on how cool AI is. Many of us want to pursue this field but are too afraid to do so because of the jargon it has like Backpropagation, GAN, neural nets. But to truly understand AI you need to recognize these jargons.
So, in this article, I will try to break some of these things down in the simplest manner possible, so that by the end of this article you will know why AI is important and why you should provide AI capabilities to your products, what are different subfields of AI and the fundamental concepts of Applied Machine Learning.
So Let’s Start with “WHY”. Why is Artificial Intelligence important and why should you think about integrating it with your product?
- You use AI enabled applications more than you know. Alexa, Siri, Google Assistant are the more obvious examples of such applications you use on a daily basis. Such application uses your input as feedback and train itself for better and more accurate output/predictions. AI is now being used in every industry, some of which are Retails, Marketing, Sports, IoT etc.
- By integrating AI with your already existing SAAS products, you give them Intelligence. AI is never a stand-alone product. You improve your product by providing it with AI capabilities. AI bots, Automation, Ad providers can be clubbed with large user data to improve the performance of the tech.
- It gets better with time. AI learns through progressive algorithms such as backpropagation with we will discuss later in this article. It becomes better after each interaction.
- AI knows what to do with your data better than you do. Any AI is as good as the data it is trained on. It provides a competitive edge when it comes to the user experience.
Subfields of Artificial Intelligence
To sum up AI in two components,
AI = Large Dataset + Intelligent Algorithms
It learns from the patterns or features in the data with the help of progressive algorithms. AI is a broad field of study that includes many theories, methods, and technologies, as well as the following major subfields:
- Machine Learning: Artificial Intelligence being the superset, Machine learning is its subset, an application of AI that meant to provide machines with human-like capabilities to learn and improve with experience. The learning process starts with data observations just like any human learn and that is through experience and more experience means more expertise. In the case of machine learning, more experience means more data.
- A neural network is yet another part of machine learning and as the very name suggest neural net is nothing but a network of interconnected units called neurons. Neurons are nothing but some function that takes an input and produces an output based on the function think of f(x) = y where f is the function/neuron, a network of such functions that produces an output after considering different functions. This is the simplest way you can understand what a neural network is, there is a lot more to it.
- Deep learning leverages on the fact that we now have fast and ample computing power and is built over vast multi-layered neural networks. That improves the accuracy and help learn more complex patterns in large amounts of data. Deep learning is most commonly used for images and speech recognition.
- Cognitive computing thrives to make the interaction between human and machine more natural, it is yet another subfield of AI. The goal is to simulate human processes in machines. The most common example of cognitive computing is AI enabled robots.
- Computer vision is a combination of deep learning and pattern recognition in images. The ultimate goal through computer vision is for a machine to process and interpret an image and videos in real time and understand its surroundings.
- Natural language processing (NLP) Through NLP we are trying to provide machine the ability not just to understand human language but to analyze and reproduce speech.
Applied Machine Learning Process
Once you get into solving Machine Learning problems, you will start identifying the pattern that you can apply on all your ML projects.
In this section of the article, I have shared a process that can be used as the kick-off or template for your next Machine learning project
Always start with “WHY”
Before solving any ML problem you need to DEFINE the problem.
Defining an ML problem
Step 1 Why does it need to be solved?
- Find the motivation
For example, you might be trying to solve a problem as a learning experience or to improve the user experience of your existing products.
Step 2 What are the benefits of the solution?
- Consider the benefits of having the problem solved
- It is important to be clear on the benefits of the problem being solved to ensure that you capitalize on them. These benefits can be used to sell the project to colleagues and management to get buy-in and additional time or budget resources
Step 3 Define the steps to solve the problem
- Design the solution, what kind of data will be collected for the related problem, the nature of your solution(Web App, just Model, open APIs).
Every Machine Learning Solution starts with Data Collection. This part involves the visualization of the data, More is always better, In Machine Learning, larger the dataset, better will be your accuracy.
Cleaning The Data
Not all datasets are as clean and pretty as you can find on Kaggle, in real life data is always messy and any Machine Learning model is as good as the data it is getting trained on. Always try to reach the maximum sophisticate in your data, Once the data is collected, the next step is to clean your data as much as possible and you can achieve that by following some rules
- Try converting all your data in the same format/type. An example would be, let’s say you have to solve an NLP problem and you have a CSV text data, some of which are in lowercase some are uppercase and some have its first letter capital try converting all of it to lower case
- Eliminate any irrelevant and common features
Luckily we live in the age of open source, there are many Python open-source libraries available the makes this mundane task much more bearable. Some of which are listed below.
These Libraries have different use cases. You can find one which solves your use-case.
Earlier, I’d said that more is always better when it comes to Machine Learning, but here is a little twist, more data can also result in more computational and memory requirements. So it’s always better to have a smaller set of your data, a sample from your data that you can test on your algorithm. This will be much faster and computationally less expensive before you go for the complete dataset.
Your machine understands data in forms of numbers. Similarly, input for any Machine learning model will always be in numerical form. So you always need to make sure that your data representation is such that your algorithm can understand.
Now, finally, it’s time to get some actual results from those data by feeding it to your Algorithm. But wait, is that all? Nope.
Feeding those data to any machine learning algorithm will give you the results, but does it mean that it has the highest accuracy rate? No, definitely not.
Before choosing your algorithm you must try to feed your sampled data to as many machine learning algorithms as you can. That will help you get the highest accuracy rate amongst all.
This way, you can evaluate the most suitable Machine Learning algorithm for your dataset. We will have a separate blog on How to Measure the Efficiency of a Machine Learning Algorithm. So, stay tuned for that.