Studying adaptive optimization | #13

Finding the top when everything is changing

2023-08-26: Greetings readers. The night air is cooling in Santa Fe and at home are focused on building a garden to produce food from the once deserted backyard. Besides that it is work, family, family, science, sleep, repeat for yours truly. Another busy week. I very much enjoyed the commentary from readers in my last update. The jury is still out on whether mind over matter is a matter of fact, or just pseudo scientific whimsy. Who knows? The line between certainty and impossibility is a fun space to explore curiosities; perhaps even do solid research. Or make wild art.

Early evening view

I’ve changed up my academic schedule for this semester and opted to replace Modern Complexity Theory with a higher level graduate course called Foundations of Adaptive Optimization, taught by Dr. Sara S. Lam. Hiroki Sayama suggested I focus on upper level course with research focus and this is a nice one with lots of practical applications for my research and work.

The point of this course is to develop creative ways of finding the optimal solutions when there is just too much darn data to iterate through every possibility. In those situations there’s inherently this question of “have we reached the top?” The “top” in this sense is often metaphorical - a mathematical formulation to represent an objective we’re trying to maximize or minimize.

Sometimes this objective is straightforward; for example in my work managing data often the goal is to minimize cost of infrastructure without giving up too much performance. But how about those examples where we’re trying to maximize team satisfaction? This kind of problem is much more art than science (but science can be a guide).

Nature is an inspiration for me. Life has a propensity to optimize for survival, and in doing so all sort of creative approaches to coordinating efficiently emerge. I want to learn from those nature-based systems for my work in information and people leadership.

Nature is also a complex adaptive system, always changing, always evolving. So that adds an additional wrinkle to staying on top - what is the algorithm to not only find the best, but stay there as the landscape changes?

That what I’ll be learning about over the next few months, and I’ll be working on some research methods to explore problems that I find interesting in this space.

Do you have any problems in your domain that you think we can take inspiration from nature to learn about?