Consider the situation of an American businessman early in the [20th] century who is interested in making money by selling Arab oil in the United States. The first problem is designing the chemical process needed to convert the type of oil in Saudi Arabia to the kind of gasoline needed in the U.S., a problem that has been solved by a French chemist. The American businessman needs to determine if global energy prices will rise to the point to make the enterprise profitable, a problem that is well understood by an English economist. The economist needs to understand how this new source of oil will affect the other oil markets, which can be solved by a Russian mathematician.
It turns out that the easiest part of this problem is translating American English to British English to Russian to French to Arabic. Much more intractable is the problem of translating between the fields of business, economics, mathematics, chemistry and the oil industry. Particularly in today’s relatively specialized economy, it is difficult to get people with different training to understand the subtle languages of different fields.
…
We propose the following starting axioms in the modeling process:
- Data is not correct, because humans are inherently tolerant of bad data.
- Operations people cannot describe what they are doing or why.
- Management does not understand their business, and cannot articulate with any accuracy how the operations people are solving the problem, or how they should solve the problem (but they believe they can).
- No single group of people can agree on what is best.
- Almost everyone can contribute an important piece of some part of the problem.
…
Once a simple model is up and running, it is important to focus on basics such as
the accuracy of the data coming to the system and the accuracy of our modeling of
the physics of the problem, and ignore in the first iterations the actual quality of the
solution. In most cases, the solution will be poor, but not because of the algorithm.
After iterating on the data requirements and the physical process, the time will come
when the data appears to be right, but the answer is not very good, and a user can
find better solutions than the computer using only the information available to the
computer. This is the signal that an advancement in the search procedure is needed.
This process then continues indefinitely (we assume that the problem is sufficiently
complex that it can always be improved).This strategy suggests the following process of implementing and improving a
model:
- Start with the simplest possible model that represents a reasonable starting
point to begin identifying data problems.- Develop the model quickly to solicit user inputs as fast as possible.
- Design the software so that it is easy to change.
2 Likes
This is how I view the problem as well, going for a simple model to a more complex one.
The question is this one:
What kind of tools do we need to develop that make this update possible. I am especially worried for the cases where we have erroneous data / predictions derived from the previous model. How can we move on?
On a different note, I believe that @clemens and Michel Beaudouin-Lafon notion of “tools” that help the user perform a goal is related to the above. We can switch between using tools that the user uses to perform a task, to removing the user entirely.
And we need to be able to switch to tools whenever the model does not work. A single executable should be able to be split into multiple tools that a user can apply manually.
1 Like