Chomsky Hierarchy in Theory of Computation

 

Types of Complexity Classes | P, NP, CoNP, NP hard and NP complete

 

Named Entity Recognition

 

Plot of the univariate Gaussian showing the mean μ and the standard deviation σ.

 

The concept of probability for discrete variables can be extended to that of a probability density p(x) over a continuous variable x and is such that the probability of x lying in the interval (x, x+δx) is given by p(x)δx for δx → 0. The probability density can be expressed as the derivative of a cumulative distribution function P(x).

 

An illustration of a distribution over two variables, X, which takes 9 possible values, and Y , which takes two possible values. The top left figure shows a sample of 60 points drawn from a joint probability distribution over these variables. The remaining figures show histogram estimates of the marginal distributions p(X) and p(Y ), as well as the conditional distribution p(X|Y = 1) corresponding to the bottom row in the top left figure.

 

Plots of the solutions obtained by minimizing the sum-of-squares error function using the M = 9 polynomial for N = 15 data points (left plot) and N = 100 data points (right plot). We see that increasing the size of the data set reduces the over-fitting problem.

 

MLOps

 

Simplified depiction of how a data change in one block of an existing chain would require changing the hashes in subsequent blocks. This would be an exceptionally challenging task

 

Simplified depiction of block design and mechanisms of linking blocks using hashes

 

An overview of AutoML pipeline covering data preparation, feature engineering, model generation and model evaluation.

 

Overview of the model-development process.

 

The lifecycle of a predictive model. defining

 

Shift in the relative importance and effort (symbolically represented by the shaded boxes) put in different phases of data-driven modelling.

 

A good scientific talk is a good scientific story

 

A typical plot diagram for an action movie can be similar to a the narrative structure of a scientific presentation. Just as a protagonist overcomes obstacles through a series of action scenes, a scientist pursues scientific objectives through a series of experiments to reach a goal. As a scientist comes closer and closer to reaching a goal, curiosity comes to a maximal point, the initial scientific question is  answered, and the presentation reaches a sense of resolution.

Machine learning used in process of data science

 

Top non-technical nuggets of wisdom of code

•  Your code should work first; you can worry about optimization later.
• Your code needs to run on a machine, but it also needs to be read by a human.
• Keep away from boxes; you’re a developer.
• Write unit tests for your code!
• Knowledge is everywhere; learn how to tap into it and make the most of it.
• Side projects are fantastic and should not be a scary thing to tackle.
• Avoid companies that are like family.
• Unlimited time off can be a trap, but it can also be a wonderful perk; make sure you ask the right questions.
• Trivial perks that only adorn the job offering, such as free food and chill-out zones should be ignored.
• Flexible hours, paid parental leave, company gear, and other useful perks are what you should look for as part of a job offer.
• Plan first, code later. That way you have a blueprint to base your work on.
• Everyone in your team is as important as you, even those who don’t write code.
• Your ego can be the end of your career; you have to learn to keep it under control.
• Do not fear code reviews. They’re a perfect learning opportunity; learn to get the most out of them.

Attention Rebellion

• Cause One: The Increase in Speed, Switching, and Filtering
• Cause Two: The Crippling of Our Flow States
• Cause Three: The Rise of Physical and Mental Exhaustion
• Cause Four: The Collapse of Sustained Reading
• Cause Five: The Disruption of Mind-Wandering
• Cause Six: The Rise of Technology That Can Track and Manipulate You
• Cause Seven: The Rise of Cruel Optimism
• Cause Eight: The Surge in Stress and How It Is Triggering Vigilance
• Causes Nine and Ten: Our Deteriorating Diets and Rising Pollution
• Cause Eleven: The Rise of ADHD and How We Are Responding to It
• Cause Twelve: The Confinement of Our Children, Both Physically and Psychologically

Complex Is Better Than Complicated

The Zen of Python—a collection of principles that summarize the core philosophy of the language—has crystal-clear points like these:

• Simple is better than complex.
• Complex is better than complicated.
• Flat is better than nested.
• Readability counts.

If the implementation is easy to explain, it may be a good idea.

DATA INTEGRITY

Data integrity refers to the physical characteristics of collected data that determine the reliability of the information. Data integrity is based on parameters such as completeness, uniqueness, timeliness, accuracy, and consistency.

Completeness

Data completeness refers to collecting all items necessary to the full description of the states of a considered object or process. A data item is considered complete if its digital description contains all attributes that are strictly required for human or machine comprehension. In other words, it may be acceptable to have missing pieces in the expected records (i.e., no contact information) as long as the remaining data is comprehensive enough for the domain.

For example, when a sensor (e.g., an IoT sensor) is involved, you might want it to sample data at a frequency of 10 minutes or even less if required or appropriate for the scenario. At the same time, you might want to be sure that the timeline is continuous with no gaps in between. If you plan to use that data to predict possible hardware failures, then you need be sure you can keep an eye close enough to the target event and not miss anything along the way.

Completeness results from having no gaps in the data from what was supposed to be collected and what is actually collected. In automatic data collection (i.e., IoT sensors), this aspect is also related to physical connectivity and data availability.

Uniqueness

When large chunks of data are collected and sampled for further use, there’s the concrete risk that some data items are duplicated. Depending on the business requirements, duplicates may or may not be an issue. Poor data uniqueness is an issue if, for example, it could lead to skewed results and inaccuracies.

Uniqueness is fairly easy to define mathematically. It is 100 percent if there are no duplicates. The definition of duplicates, however, depends on the context. For example, two records about Joseph Doe and Joe Doe are apparently unique but may refer to the same individual and then be duplicates that must be cleaned.

Timeliness

Data timeliness refers to the distribution of data records within an acceptable time frame. The definition of an acceptable time frame is also context-specific. It refers to the duration of the time frame and the appropriate timeline.

In predictive maintenance, for example, the timeline varies depending on the industry. Usually, a 10-minute timeline is more than acceptable but not for reliable fault predictions in wind turbines. In this case, a 5-minute interval is debated, and some experts suggest an even shorter rate of data collection.

Duration is the overall time interval for which data collection should occur to ensure reliable analysis of data and satisfactory results. In predictive maintenance, an acceptable duration is on the order of two years’ worth of data.

Accuracy

Data accuracy measures the degree to which the record correctly describes the observed real-world item. Accuracy is primarily about the correctness of the data acquired. The business requirements set the specifications of what would be a valid range of values for any expected data item.

When inaccuracies are detected, some policies should be applied to minimize the impact on decisions. Common practices are to replace out-of-range values with a default value or with the arithmetic mean of values detected in a realistic interval.

Consistency

Data consistency measures the difference between the values reported by data items that represent the same object. An example of inconsistency is a negative value of output when no other value reports failures of any kind. Definitions of data consistency are, however, also highly influenced by business requirements.