The Good and the Bad News for Artificial Expertise

Algorithmic Vs. Al Problem Solving

Spiral Model

The Waterfall Cycle Never Exists! (notice the backward arrows)

Iterative development produces the architecture first,

Effective Deployment of 6 Best Practices

The Rational Unified Process describes how to effectively deploy commercially proven approaches to software development for software development teams. These are called “best practices” not so much because you can precisely quantify their value, but rather, because they are observed to be commonly used in industry by successful organizations. The Rational Unified Process provides each team member with the guidelines, templates and tool mentors necessary for the entire team to take full advantage of among others the following best practices:

1. Develop software iteratively
2. Manage requirements
3. Use component-based architectures
4. Visually model software
5. Verify software quality
6. Control changes to software

Develop Software Iteratively  Given today’s sophisticated software systems, it is not possible to sequentially first define the entire problem, design the entire solution, build the software and then test the product at the end. An iterative approach is required that allows an increasing understanding of the problem through successive refinements, and to incrementally grow an effective solution over multiple iterations. The Rational Unified Process supports an iterative approach to development that addresses the highest risk items at every stage in the lifecycle, significantly reducing a project’s risk profile. This iterative approach helps you attack risk through demonstrable progressfrequent, executable releases that enable continuous end user involvement and feedback. Because each iteration ends with an executable release, the development team stays focused on producing results, and frequent status checks help ensure that the project stays on schedule. An iterative approach also makes it easier to accommodate tactical changes in requirements, features or schedule.

Manage Requirements  The Rational Unified Process describes how to elicit, organize, and document required functionality and constraints; track and document tradeoffs and decisions; and easily capture and communicate business requirements. The notions of use case and scenarios proscribed in the process has proven to be an excellent way to capture functional requirements and to ensure that these drive the design, implementation and testing of software, making it more likely that the final system fulfills the end user needs. They provide coherent and traceable threads through both the development and the delivered system.

Use Component-based Architectures  The process focuses on early development and baselining of a robust executable architecture, prior to committing resources for full-scale development. It describes how to design a resilient architecture that is flexible, accommodates change, is intuitively understandable, and promotes more effective software reuse. The Rational Unified Process supports component-based software development. Components are non-trivial modules, subsystems that fulfill a clear function. The Rational Unified Process provides a systematic approach to defining an architecture using new and existing components. These are assembled in a well-defined architecture, either ad hoc, or in a component infrastructure such as the Internet, CORBA, and COM, for which an industry of reusable components is emerging.

Visually Model Software  The process shows you how to visually model software to capture the structure and behavior of architectures and components. This allows you to hide the details and write code using “graphical building blocks.” Visual abstractions help you communicate different aspects of your software; see how the elements of the system fit together; make sure that the building blocks are consistent with your code; maintain consistency between a design and its implementation; and promote unambiguous communication. The industry-standard Unified Modeling Language (UML), created by Rational Software, is the foundation for successful visual modeling.

Verify Software Quality  Poor application performance and poor reliability are common factors which dramatically inhibit the acceptability of today’s software applications. Hence, quality should be reviewed with respect to the requirements based on reliability, functionality, application performance and system performance. The Rational Unified Process assists you in the planning, design, implementation, execution, and evaluation of these test types. Quality assessment is built into the process, in all activities, involving all participants, using objective measurements and criteria, and not treated as an afterthought or a separate activity performed by a separate group.

Control Changes to Software  The ability to manage changemaking certain that each change is acceptable, and being able to track changesis essential in an environment in which change is inevitable. The process describes how to control, track and monitor changes to enable successful iterative development. It also guides you in how to establish secure workspaces for each developer by providing isolation from changes made in other workspaces and by controlling changes of all software artifacts (e.g., models, code, documents, etc.). And it brings a team together to work as a single unit by describing how to automate integration and build management.

Six Sigma

Earned Value System

Cleanroom Software Engineering

K-means: Common choices for proximity, centroids, and objective functions.

Different types of clusters as illustrated by sets of two-dimensional points.

Agglomerative Hierarchical Clustering

Full Stack

General Approach to Solving a Classification Problem

A classification technique (or classifier) is a systematic approach to building classification models from an input data set. Examples include decision tree classifiers, rule-based classifiers, neural networks, support vector machines, and na ̈ıve Bayes classifiers. Each technique employs a learning algorithm to identify a model that best fits the relationship between the attribute set and class label of the input data. The model generated by a learning algorithm should both fit the input data well and correctly predict the class labels of records it has never seen before. Therefore, a key objective of the learning algorithm is to build models with good generalization capability; i.e., models that accurately predict the class labels of previously unknown records. 

Da imobilização à internalização da mudança

Classification

Classifying an unlabeled vertebrate. The dashed lines represent the outcomes of applying various attribute test conditions on the unlabeled vertebrate. The vertebrate is eventually assigned to the Non-mammal class. 

Training and test error rates

Notice that the training and test error rates of the model are large when the size of the tree is very small. This situation is known as model underfitting. Underfitting occurs because the model has yet to learn the true structure of the data. As a result, it performs poorly on both the training and the test sets. As the number of nodes in the decision tree increases, the tree will have fewer training and test errors. However, once the tree becomes too large, its test error rate begins to increase even though its training error rate continues to decrease. This phenomenon is known as model overfitting. 

The input data for a classification task

The input data for a classification task is a collection of records. Each record, also known as an instance or example, is characterized by a tuple (x, y), where x is the attribute set and y is a special attribute, designated as the class label (also known as category or target attribute).

Characteristics of Decision Tree Induction

1. Decision tree induction is a nonparametric approach for building classification models. In other words, it does not require any prior assumptions regarding the type of probability distributions satisfied by the class and other attributes.
2. Finding an optimal decision tree is an NP-complete problem. Many decision tree algorithms employ a heuristic-based approach to guide their search in the vast hypothesis space.
3. Techniques developed for constructing decision trees are computationally inexpensive, making it possible to quickly construct models even when the training set size is very large. Furthermore, once a decision tree has been built, classifying a test record is extremely fast, with a worst-case complexity of O(w), where w is the maximum depth of the tree.
4. Decision trees, especially smaller-sized trees, are relatively easy to interpret. The accuracies of the trees are also comparable to other classification techniques for many simple data sets.
5. Decision trees provide an expressive representation for learning discrete-valued functions. However, they do not generalize well to certain types of Boolean problems. One notable example is the parity function, whose value is 0 (1) when there is an odd (even) number of Boolean attributes with the value T rue. Accurate modeling of such a function requires a full decision tree with 2d nodes, where d is the number of Boolean attributes.
6. Decision tree algorithms are quite robust to the presence of noise, especially when methods for avoiding overfitting are employed.
7. The presence of redundant attributes does not adversely affect the accuracy of decision trees. An attribute is redundant if it is strongly correlated with another attribute in the data. One of the two redundant attributes will not be used for splitting once the other attribute has been chosen. However, if the data set contains many irrelevant attributes, i.e., attributes that are not useful for the classification task, then some of the irrelevant attributes may be accidently chosen during the tree-growing process, which results in a decision tree that is larger than necessary. Feature selection techniques can help to improve the accuracy of deci- sion trees by eliminating the irrelevant attributes during preprocessing.
8. Since most decision tree algorithms employ a top-down, recursive partitioning approach, the number of records becomes smaller as we traverse down the tree. At the leaf nodes, the number of records may be too small to make a statistically significant decision about the class representation of the nodes. This is known as the data fragmentation problem. One possible solution is to disallow further splitting when the number of records falls below a certain threshold.
9. A subtree can be replicated multiple times in a decision tree. This makes the decision tree more complex than necessary and perhaps more difficult to interpret. Such a situation can arise from decision tree implementations that rely on a single attribute test condition at each internal node. Since most of the decision tree algorithms use a divide-and-conquer partitioning strategy, the same test condition can be applied to different parts of the attribute space, thus leading to the subtree replication problem.
10. The test conditions described so far involve using only a single attribute at a time. As a consequence, the tree-growing procedure can be viewed as the process of partitioning the attribute space into disjoint regions until each region contains records of the same class. The border between two neighboring regions of different classes is known as a decision boundary. Since the test condition involves only a single attribute, the decision boundaries are rectilinear; i.e., parallel to the “coordinate axes.” This limits the expressiveness of the decision tree representation for modeling complex relationships among continuous attributes.

General approach for building a classification model

Classification as the task of mapping an input attribute x into its class label y

The Success of Decision Tree

The success of decision trees is explained by several factors that make them quite attractive in practice:

• Decision trees are non-parametric. They can model arbitrarily complex relations between inputs and outputs, without any a priori assumption;
• Decision trees handle heterogeneous data (ordered or categorical variables, or a mix of both);
• Decision trees intrinsically implement feature selection, making them robust to irrelevant or noisy variables (at least to some extent);
• Decision trees are robust to outliers or errors in labels;
• Decision trees are easily interpretable, even for non-statistically oriented users.

The Five Tribes of Machine Learning, and What You Can Take from Each

A Gentle Introduction to Random Forests

The random forest (Breiman, 2001) is an ensemble approach that can also be thought of as a form of nearest neighbor predictor. Ensembles are a divide-and-conquer approach used to improve performance. The main principle behind ensemble methods is that a group of “weak learners” can come together to form a “strong learner”. The figure below (taken from here) provides an example. Each classifier, individually, is a “weak learner,” while all the classifiers taken together are a “strong learner”.

Developers, Designers and Project Managers

Angra - Carolina IV

The Five Tribes of Machine Learning

Where and When to Publish?

Escrevendo um Abstract

1. Contexto. Nessa parte, você tem que apresentar ao leitor do que se trata o seu trabalho, isto é, contextualizá-lo. Uma forma simples de fazer isso é responder, em poucas frases, as seguintes perguntas: Qual o problema que você está investigando? Onde esse problema se apresenta? Por que esse problema é importante? Lembre que são nessas primeiras linhas que você vai despertar o interesse do leitor.
2. Objetivo. Descreva o objetivo do seu trabalho em relação a problemática apresentada no Contexto. Por exemplo, você pode dizer que o objetivo é compreender um aspecto do problema, propor uma solução para o problema, realizar um estudo de caso, dentre outros. O texto do objetivo vai depender do tipo de estudo que você realizou.
3. Método. Aqui você descreve sucintamente quais foram os métodos utilizados; se foi um estudo de caso, experimento, revisão de literatura, etc. É bom colocar alguns detalhes do método, como por exemplo a quantidade de participantes ou número de estudos analisados. Isso poderá despertar um maior interesse por seu trabalho.
4. Resultados. Você deve apresentar os principais resultados do seu estudo, de forma bem sucinta. Se atenha aos resultados mais relevantes do seu estudo. Lembre que você quer informar e também despertar o interesse do leitor. Sempre seja sincero quanto aos resultados, não importa se eles são favoráveis ou não ao seu trabalho.
5. Conclusão. Na conclusão é importante que você descreva quais foram as principais descobertas dos seu estudo. Por exemplo, você pode descrever quais são as implicações do seu estudo para o estado da arte e da prática, ou como o estudo pode servir como base para trabalhos futuros. Não seja exaustivo, descreva apenas os principais pontos.

Videoclipe # 20 Carta aos Missionários - Uns e Outros

Guns N' Roses - Catcher In The Rye

“Se a gente faz uma coisa bem demais, aí, depois de algum tempo, se não tiver muito cuidado, começa a se exibir. E aí a gente deixa de ser bom de verdade.” –
O apanhador no campo de centeio

O Jogo da Vida Real

The conceptual goals of various recommender systems

Comparing the traditional classification problem with collaborative filtering

Examples of utility matrices

Example of 5-point interval ratings

Paradigms of recommender systems

Recommender system as a black box

A Taxonomy for Artificial and Computational Intelligence

Recommender Systems An Introduction

In this age of information overload, people use a variety of strategies to make choices about what to buy, how to spend their leisure time, and even whom to date. Recom- mender systems automate some of these strategies with the goal of providing afford- able, personal, and high-quality recommendations. This book offers an overview of approaches to developing state-of-the-art recommender systems.

Binning

Binning or discretization is the process of transforming numerical variables into categorical counterparts. An example is to bin values for Age into categories such as 20-39, 40-59, and 60-79. Numerical variables are usually discretized in the modeling methods based on frequency tables (e.g., decision trees). Moreover, binning may improve accuracy of the predictive models by reducing the noise or non-linearity. Finally, binning allows easy identification of outliers, invalid and missing values of numerical variables.

Down on the Farm

The (flawed) comparison between brain and computer at a glance

Terms to know

1. Motive: a need or desire that influences behavior
2. Speculative statement: a statement of something that could logically happen in the present circumstances.
3. Point of View: an attitude or feeling about an issue
4. Theory: an explanation that may grow out of an hypothesis; tries to explain a wide variety of circumstances
5. Bias: a slant in favor or against
6. Fact: something that can be proven
7. Precedent: examples of earlier actions or events that can be used as an example in a later action
8. Opinion: something that cannot be proven; usually involves ones feelings about a subject
9. Objection: a statement against a statement, theory, or action.
10. Belief: a thought held to be true no matter what the facts are
11. Consequences: those actions or reactions that happen as a result of something being done or said

Commons Ways to Use Propaganda

Assertion
Assertion consists of simply stating a debatable idea as a fact. It relies on the premise that people like to believe what they are told. Examples include: “Women are bad drivers,” and “Men never stop to ask for directions.” Assertions are also common in advertising claims. Sometimes facts are fabricated to support a claim.

Bandwagon
The “bandwagon effect” refers to any situation in which people attempt to be part of a successful or popular endeavor merely for the sake of its popularity. Peer pressure can come into play if “everyone else is doing it.” The problem is that you do not know how other people made their decision. It may be that everyone is simply going along with what is popular. It is a natural human desire to want to be part of a majority group or a winning team.

Card Stacking 
This is where an unfair advantage is given to one point of view, while weakening another. Information may be presented out of context and important facts can be obscured. Even if alternate views are presented, they tend to be presented last and with less skill. Censorship is an extreme form. (You see this in politics a lot. The way opposite sides present their arguments on current tax legislation is a good example.)

Glittering Generalities
These are colorful vague terms that are meant to appeal to the audience. They are often left to stand alone without context or definition. The idea is to gain unquestioning approval of whatever is said. They are common in political campaigns and advertising. Look for vague, positive words that are not explained. (Hope and change were often used in the last presidential campaign.)

False Dilemma
Also known as false choice, it involves reducing a complex argument to a small number of alternatives and concluding that only one option is appropriate. “Either you agree with us, or you are a fool.” In the real world most issues are not so simple. (The Democrats might tell you that if we don’t raise taxes on the rich, the government will go broke. The Republicans might say that if we raise taxes on the rich, there will be fewer jobs.)

The Lesser of Two Evils
This is a type of false dilemma that offers two “bad” alternatives. Here the propagandist is trying to get you to adopt a perspective you would be hesitant to accept. To do so, an even worse alternative is offered as the only option. The imperfect option is better than a horrendous one. (In an environment where all politicians have low approval rates, you might be asked to vote for someone who isn’t as bad as the alternative.)

Name Calling
Name-calling is the use of negative words to disparage an enemy or an opposing view. Insulting words take the place of logical agreements, appealing to emotions, rather than reason. This is the opposite of glittering generalities.(Politicians on both sides use the word extremist to describe someone who simply disagrees with them. Republicans use the term socialist to describe people on the left, while democrats like to accuse republicans of trying to give more money to billionaires as if being one is bad.)

Pinpointing the Enemy
Problems rarely stem from a single cause, but propagandists often oversimplify. When an enemy is blamed for something that is someone else’s fault, it is a form of this technique known as scapegoating. The Nazi’s blamed the jews for Germany’s economic problems prior to World War II. Even if the target is responsible for part of the problem, they are probably not the sole cause. (Doug: Both parties blame each other for recent economic problems.)

Plain Folk
Here the propagandist tries to paint himself as just like you. Problems are blamed on outsiders. The risk is that the person using this technique is exposed as an impostor. Politicians often paint themselves as plain folk rather than Washington insiders. (Doug: Politicians on both sides claim to be in touch with the grass roots or the blue collar crowd. They want you to think they support Main Street rather than Wall Street.)

Testimonials
Most testimonials are made by famous people who tell us how much they like a product or a politician. The propagandist hopes that your feelings about the famous person will transfer to the product or cause he or she endorses. In addition to celebrity testimonials, politicians use “plain folk” testimonials where someone like you tells you why you should vote for someone. Some testimonials are not propaganda. An expert opinion from someone who does not gain financially is an example. Keep in mind that you need faulty reasoning in order to have propaganda.

Transfer
This is a subtile technique also known as association or false connection. It is often done in a symbolic manner. If you pose with the American flag you are sending the message that you are a patriot. Such use of symbols only becomes propaganda when symbols are use to send an unspoken emotional message. (Doug: Politicians will picture an opponent with someone who they think is less popular. In more liberal districts, democrats will try to associate opponents with Rush Limbaugh. Republicans often try to connect their opponents with Nancy Pelosi from the liberal hotbed of SanFrancisco.)

Desires and Fears
Most actions are motivated by desires and fears, and a skilled propagandist can use either against you. Many fears and desires are two sides of the same coin. We desire love and fear rejection. Many ads imply romance will follow if you use their product. Others imply that the product will prevent rejection. We desire prosperity and control, and fear being powerless and lacking material wealth. We desire immortality and fear death. (Doug: People put up with invasive searches at airports due to fear of terrorists. Attractive celebrities do ads for products designed to help you live longer.)

---

Positives Uses
While the word has negative connotations, propaganda can be used for good. It can be used to end slavery, promote peace, or encourage people to improve the world or their local community. It can be used to encourage and inspire. It can also engender sympathy and generosity and promote civic minded behavior. Some people disapprove of using propaganda in this manner, while others believe that the ends justify the means.

Negative Uses
Negative propaganda often promotes fear and hostility. Propaganda that promotes fear, leaves people open to further propaganda. Such fear is often disproportionate to the actual danger. Anger is closely related and used to much the same effect. It is a common reaction to injustice, humiliation, or betrayal. It shuts down rational thought. Hatred is like fear in that it is a powerful motivator of human emotions. Negative feelings can be used to promote discrimination, violence, and property violation. They can also be used to dehumanize and violate human rights. At the opposite end, they can deify.

For more go to DrDougGreen.Com

What is propaganda?

Propaganda is always manipulative. It is intended to give someone else control over your thoughts and actions. This is why it is crucial the you be able to know it when you see it so you can make rational, conscious decisions. It comes in many forms, but you can recognize it by its persuasive function, sizable target audience, the representation of a specific group’s agenda, and the use of faulty reasoning and/or emotional appeals. It is not always harmful. In some situations it can be benign or even positive.

A3 Implementation

Polemic Subject: What’s an Employee Worth?

Os 7 desperdícios na manufatura e no escritório

The Lean Production System

Hypothetical Example of Missing Data

p Values

Data, Information, Knowledge, Wisdom, and Understanding

Partitioning Services into Layers

The Smart UI “Anti-Pattern”

Advantages

• Productivity is high and immediate for simple applications.
• Less capable developers can work this way with little training.
• Even deficiencies in requirements analysis can be overcome by releasing a prototype to users and then quickly changing the product to fit their requests.
• Applications are decoupled from each other, so that delivery schedules of small modules can be planned relatively accurately. Expanding the system with additional, simple behavior can be easy.
• Relational databases work well and provide integration at the data level.
• 4GL tools work well.
• When applications are handed off, maintenance programmers will be able to quickly redo portions they can't figure out, because the effects of the changes should be localized to each particular UI.

Disadvantages

• Integration of applications is difficult except through the database.
• There is no reuse of behavior and no abstraction of the business problem. Business rules have to be duplicated in each operation to which they apply.
• Rapid prototyping and iteration reach a natural limit because the lack of abstraction limits re- factoring options.
• Complexity buries you quickly, so the growth path is strictly toward additional simple applica- tions. There is no graceful path to richer behavior.

Layer Volatility

Layering requirements derived from requirements analysis using QFD

A process of architectural design from the layered requirements

Transatlantic - Bridge Across Forever (2001)

Escalas e Operações

The Scientific Method

Música de velho