Data Science and AI Blog.
Our Data Scientist / AI Engineer is writing about
Data Science / Mathematics / Statistics / Engineering.
Do even the best DNN models understand language? New advances, new excitement Without any doubt, Deep Neural Networks (DNNs) have brought huge improvements to the NLP world recently. News like an AI model using DNN can write articles like a human or can write code to create a website like a real developer comes to …
Do even the best DNN models understand language? Read More »
Introduction In a modeling process, there are 3 core concepts that will always exist: Data. Features. Type of model and its corresponding parameters. From data to the model, features are a measurable representation of the data, which would be the format for the data to be processed by the model thus method to create features …
In a typical Machine Learning project, one would need to find out how good or bad their models are by measuring the models’ performance on a test dataset, using some statistical metrics. Various performance metrics are used for different problems, depending on what needs to be optimized by the models. For this blog, we will …
Performance Metrics for Weather Images Forecasting Read More »
Handling Missing Data – Abstract The article discusses various types of missing data and how to handle them. We demonstrate how the prediction results are affected by the quality of missing data as well as the method of handle missing data through some experiments. Introduction – Handling Missing Data For any real data set, missing data …
In this blog, we would like to introduce basic concepts in causal inference and the potential outcome framework. 1.Causality terminology Unit: The fundamental notion is that causality is tied to action (or manipulation, treatment, or intervention), applied to a unit. A unit here can be a physical object, a firm, a person, at a particular …
Causal inference and potential outcome framework Read More »
As word embeddings become more and more ubiquitous in language applications, a key issue has likewise emerged. The ability of embeddings to learn complex, underlying relationships between words is also their greatest caveat: How do we know when we have trained a good embedding? It’s important to differentiate between a good embedding in a more general sense and …
Word Embeddings – blessing or curse in disguise? Read More »
In Data Science, before building a predictive model from a particular data set, it is important to explore and perform pre-processing data. In this blog, we will illustrate some typical steps in data pre-processing. In this particular exercise, we will build a simple Decision Tree model to classify the food cuisine from the list of …
Overview for Build a Data lake with AWS(Beginner) AWS Lake Formation enables you to set up a secure data lake. A data lake is a centralized, curated, and secured repository storing all your structured and unstructured data, at any scale. You can store your data as-is, without having first to structure it. And you can run different …
Recently, I was asked to solve an interesting problem: The problem of Sorting Array: Input: An array A contains unique n-elements, whose values are integers. The length (n) of A is ranged from 2 to10. Output: A sorted array B in ascending order. The length of array B must be the same as the length …
Combinatorial Optimization: From Supervised Learning to Reinforcement Learning – Part 1 Read More »
Overview (AWS – Batch Process and Data Analytics) Today, I will describe our current system references architecture for Batch processing and Data Analytics for the sales report system. Our mission is to create a big data analytics system that interacts with Machine Learning features for insights and prediction. Nowadays, too many AI companies are researching …
