Understand the hierarchy of the Data Process

 

Kubeflow architecture

 

Model development life cycle

 

The Zen of Python, by Tim Peters

• Beautiful is better than ugly.
• Explicit is better than implicit.
• Simple is better than complex.
• Complex is better than complicated.
• Flat is better than nested.
• Sparse is better than dense.
• Readability counts.
• Special cases aren't special enough to break the rules.
• Although practicality beats purity.
• Errors should never pass silently.
• Unless explicitly silenced.
• In the face of ambiguity, refuse the temptation to guess.
• There should be one—and preferably only one—obvious way to do it.
• Although that way may not be obvious at first unless you're Dutch.
• Now is better than never.
• Although never is often better than *right* now.
• If the implementation is hard to explain, it's a bad idea.
• If the implementation is easy to explain, it may be a good idea.
• Namespaces are one honking great idea—let's do more of those!

TOGAF

 

Data-Driven Decision

 

Decision Problems

A decision problem consists of a specification of a subset of the possible instances. Given an instance, one is required to determine whether the instance is in the specified set (e.g., the set of prime numbers, the set of connected graphs, or the set of sorted sequences). For example, consider the problem where one is given a natural number, and is asked to determine whether or not the number is a prime. One important case, which corresponds to the aforementioned search problems, is the case of the set of instances having a solution; that is, for any binary relation R ⊆ {0, 1}∗ × {0, 1}∗ we consider the set {x : R(x) = ∅}. Indeed, being able to determine whether or not a solution exists is a prerequisite to being able to solve the corresponding search problem (as per Definition 1.1). In general, decision problems refer to the natural task of making a binary decision, a task that is not uncommon in daily life (e.g., determining whether a traffic light is red). In any case, in the following definition of solving decision problems, the potential solver is again a function; that is, in this case the solver is a Boolean function, which is supposed to indicate membership in a predetermined set.

Search Problems

A search problem consists of a specification of a set of valid solutions (possibly an empty one) for each possible instance. That is, given an instance, one is required to find a corresponding solution (or to determine that no such solution exists). For example, consider the problem in which one is given a system of equations and is asked to find a valid solution. Needless to say, much of computer science is concerned with solving various search problems (e.g., finding shortest paths in a graph, sorting a list of numbers, finding an occurrence of a given pattern in a given string, etc.). Furthermore, search problems correspond to the daily notion of “solving a problem” (e.g., finding one’s way between two locations), and thus a discussion of the possibility and complexity of solving search problems corresponds to the natural concerns of most people.

Polynomial time and exponential time

 Below are some common Big-O functions while analyzing algorithms.

• O(1) - constant time
• O(log(n)) - logarithmic time
• O((log(n))^c) - polylogarithmic time
• O(n) - linear time
• O(n²) - quadratic time
• O(n^c) - polynomial time
• O(cⁿ) - exponential time
• O(n!) - factorial time

(n = size of input, c = some constant)

Here is the model graph representing Big-O complexity of some functions

O(1) = O(yeah)
O(logn) = O(nice)
O(n) = O(k)
O(n^2) = O(my)
O(2^n) = O(no)
O(n!) = O(mg)
O(n^n) = O(sh*t!)