Category: General

Any topic really.

Fat tails are weird

Posted by Filip Piekniewski

If you have taken a statistics class it may have included stuff like basic measure theory. Lebesgue measures and integrals and their relations to other means of integration. If your course was math heavy (like mine was) it may have included Carathéodory's extension theorem and even basics of operator theory on Hilbert spaces, Fourier transforms etc. Most of this mathematical tooling would be devoted to a proof of one of the most important theorems on which most of statistics is based - central limit theorem (CLT). 

Central limit theorem states that for a broad class of what we in math call random variables (which represent realizations of some experiment which includes randomness), as long as they satisfy certain seemingly basic conditions, their average converges to a random variable of a particular type, one we call normal, or Gaussian. 

The two conditions that these variables need to satisfy are that they are:

  1. Independent
  2. Have finite variance

In human language this means that individual random measurements (experiments) "don't know" anything about each other, and that each one of these measurements "most of the time" sits within a bounded range of values, as in it can actually be pretty much always … Read more...

The Church of AGI

Posted by Filip Piekniewski

As I child I've been raised as a catholic and I vividly remember what it was like to believe in God and all the divine entities. It does bring certain amount of comfort to our lives, removes loneliness, gives a broader sense to existence. At some point however I started questioning the things I was told and eventually became more of a deist and at this point pretty much an atheist. I don't claim to know the answer to life universe and everything (except that it is 42!), in fact although I much prefer rational and objectivist approach to reality, I believe science as we know is still barely scratching the surface of the secrets of reality and in fact I would not exclude the possibility that those secrets are fundamentally unknowable. I actually think it is totally fine to admit that we live our lives in a world of uncertainty, with a plethora of events and processes around us we only pretend to understand. Moreover, even while being generally an atheist I'm willing to admit that lots of stories and rules originating in religious texts have some level of universality, especially if some of them survived for thousands of … Read more...

The Atom of Intelligence

Posted by Filip Piekniewski

Back in a very distant past, perhaps over 2 billion years ago, a wonderful thing happened: a strand of nucleic acid found itself encapsulated in a little protein bubble, along with a few other ingredients sufficient for it to replicate. This in fact may have happened millions of times before, each time dying out after a few generations. But one such bubble that appeared that day in the primordial sea was going to survive, this was the one which was going to make it and launch an incredible evolutionary process lasting until this day. A process that managed to create incredibly complex beings including you and me. 


As soon as this bubble of life started to replicate, the process "guiding" its evolution "noticed" that there are effectively two aspects of control necessary for survival:

  • Internal control regulating the expression of genetic code and other internal reactions - we call this process metabolism today.
  • External control regulating interaction of the bubble with the surrounding environment - we call the process behavior today.

Initially both control mechanisms were likely only tuned at a generational level, not so much at the level of life of an individual, but clearly an evolutionary pressure to … Read more...

Science, dogma and mysteries.

Posted by Filip Piekniewski

I was raised in a rational family, with strong belief in science. When I got my masters degree a was pretty much convinced that we generally know everything about the world and that science is more or less a complete endeavor. By the time I got my PhD however, my confidence dropped quite significantly. Now almost 13 years after my PhD defense, my view is that science is actually a rather fragile thread we use to hold together and explain various mysteries in the world. And that is not to say science is not the right method - it is! But I now view science as any other social activity, being influenced by zeitgeist, politics, fashion, financing and often stuck in a dogma, no different than the dogma that threatened Galileo or Copernicus. In fact in many ways, contrary to popular belief, I believe todays science is a lot more dogmatic than in the early XX century and probably worse than it's been during the enlightenment. Let me discuss a few areas where in my opinion the mainstream science is stuck in a dogma and let me highlight some interesting alternative theories that may be able to challenge the status … Read more...

What actually is statistics?

Posted by Filip Piekniewski

In the modern era of computers and data science, there is a ton of things discussed that are of "statistical" nature. Data science essentially is glorified statistics with a computer, AI is deeply statistical at its very core, we use statistical analysis for pretty much everything from economy to biology. But what actually is it? What exactly does it mean that something is statistical? 

The short story of statistics

I don't want to get into the history of statistical studies, but rather take a birds eye view on the topic. Let's start with a basic fact: we live in a complex world which provides to us various signals. We tend to conceptualize these signals as mathematical functions. A function is the most basic way of representing a fact that some value changes with some argument (typically time in physical world). We observe these signals and try to predict them. Why do we want to predict them? Because if we can predict a future evolution of some physical system, we can position ourselves to extract energy from it when that prediction turns out accurate [but this is a story for a whole other post]. This is very fundamental, but in principle … Read more...

Civilization from scratch

Posted by Filip Piekniewski

This post is not about AI and not about winter. I have a few of those coming, but this one is about something different. I hope you don't mind.

A friend of mine recently gave a lot to think about by stating the following thought experiment:

Imagine you are taken back in time. To what extent would you be able to advance the civilization of the given era with all the knowledge in your head (no notebooks).

Initially the reaction is obviously that since we all live and breathe the current technical civilization, one should be able to recover almost everything right? There are some many uncertainties to which we already know the answers, so this should be much easier than to get there without such insight?

When you actually give some thought to it, you will realize that things may not be so easy. First of all, in most cases if somebody was taken back in time but left in the same place, they would end up in a middle of nowhere and would have to first survive to even get into contact with any contemporary humans. Say San Diego 300 years ago was an empty costal desert, and … Read more...

A brief story of Silicon Valley's affair with AI

Posted by Filip Piekniewski

Once upon a time, in the 1980's there was a magical place called Silicon Valley. Wonderful things were about to happen there and many people were about make a ton of money. These things were all related to the miracle of a computer and how it would revolutionize pretty much everything.

Computers had a ton of applications in front of them: completely overhauling office work, enabling entertainment via computer games and changing the way we communicate, shop and use banking system. But back then they were clumsy, slow and expensive. And although the hope was there, many of these things wouldn't be accomplished unless computers somehow got orders of magnitude faster and cheaper.

But there was the Moore's law - over the decade of the 1970' the number of transistors in an integrated circuit doubled every ~18 months. If this law were to hold, the future would be rosy and beautiful. The applications would be unlocked for which the markets were awaiting. Money was to be made.

By mid 1990's it was clear that it worked. Computers were getting faster and software was getting more complex so rapidly, that upgrades had to happen on a yearly basis to keep up … Read more...

Elon and the collective

Posted by Filip Piekniewski

Elon Musk is a polarizing figure. His ideas frequently come about in casual conversations. People are often amused and impressed by his achievements. I must admit, a few years back I thought he is literally the next Steve Jobs, only actually better, since he was onto so many things... I admired SpaceX, thought that Tesla cars had many great solutions in them...

At some point in 2015 or 2016 Elon started talking outrageous stuff in the domain of AI, a domain of my own expertise, which I could tell right away was total bullshit. And then I began looking at all this stuff in detail. Doing some math here and there. Reading various opinions. As a result, my opinion on Musk and many of his ideas has changed somewhat substantially. At this point, I can pretty much say with confidence that 90% of his stuff is utter BS, and the remaining 10% is perhaps impressive but still questionable.

Nevertheless he is quite a character with many fans almost religiously believing everything he says. Any time I meet somebody who is a Musk fan I have to go over these issues so I decided to write this post as a point … Read more...

Are black holes really so black?

Posted by Filip Piekniewski

I have long been fascinated with the mysterious black holes. Over the years I've been following the literature and improved my mathematical skills to better understand what we know about these objects. Over the past several years I followed several heated debates related to numerous paradoxes that our understanding of black holes had caused. Here I'd like to present a few issues I have with our contemporary understanding of the subject. If you are a black hole specialist, I will appreciate feedback.

Classical picture

Existence of black holes is a straightforward result of the theory of general relativity (in fact is conceivable even in the classical Newtonian mechanics). In essence the observation is that an object dense enough would eventually reach the escape velocity equal to the speed of light, at which point in becomes black (since it cannot radiate anything out) and anything that happens to get trapped inside it, has no hope of getting out, or at least has the same hope of getting out as we may have the hope of traveling faster than light. The solution of that particular object was first put forward by Karl Schwarzschild who observed that there is a particular size/radius below … Read more...

Electric shock

Posted by Filip Piekniewski

Electric cars are great. They don't pollute, drive without making noise, have incredible responsiveness and torque all over the RPM range. There are limited number of moving parts, they don't need lubrication hence don't consume oil.

These are all true. There is no point in arguing with these facts, anyone who ever driven an electric car will concur. But there is always the other side, the one enthusiasts will not want to discuss. Let me go into a few issues I have with this technology.

Battery

All those amazing cars (such as Tesla) are based on Lithium Ion battery. Much like any other battery, this one uses electrodes, one made of lithium compound and the other out of a form of carbon such as graphite. The electrolyte in between these electrodes typically contains cobalt (typically in the form of an oxide). The exact chemistry varies between different types of cells but overall positively charged lithium ions get carried from the anode to cathode during discharge and the reverse is happening during charging. Cobalt oxide mediates the ions. So in some sense the electric car actually has zillions of moving parts if we count all these ions traveling from anode to … Read more...

Optimality, technology and fragility.

Posted by Filip Piekniewski

This post is a bit of a mixed bag about technology and fragility, a bit about AI and tiny bit on politics. You've been warned.

Intro

Back in the communist and then early capitalist Poland, where I grew up, one could often get used soviet equipment such as optics, power tools etc. Back in the day these things were relatively cheap and had the reputation of being very sturdy and essentially unbreakable (often described as pseudo Russian "gniotsa nie łamiotsa" which essentially meant you could "bend it and it would not break"). There are multiple possible reasons why that equipment was so sturdy, one hypothesis is that soviet factories could not control very well the quality of their steel and so the designers had to put in additional margin into their designs. When the materials actually turned out to be of high quality, such over engineered parts would then be extra strong. Other explanation is that some of that equipment was ex-military and therefore designed with an extra margin. Nevertheless, these often heavy and over-engineered products were contrasted in the early 90's with modern, optimized, western made things. Western stuff was obviously better designed and optimized, lighter, but as soon … Read more...

Introduction

Posted by Filip Piekniewski

Hello Internet,

Let me start this blog with a short introduction. My name is Filip Piekniewski and I'm a researcher working on topics of artificial intelligence, machine learning, perception etc (check my website for more info). For the past six+ years I've been working at Brain Corporation in San Diego. The company has ambitious goal of building brains for robots and the work we've been doing is quite unique. I'd like to use this blog to share some of my thoughts on Machine Learning, from a slightly different perspective than a lot of the mainstream, namely from the perspective of actually applying these techniques to a physical device existing in physical reality. As we've learned, this is a whole different ballpark than running your algorithm on a dataset in a sterile, digital world. I hope you will find this read entertaining.

 

If you found an error, highlight it and press Shift + Enter or click here to inform us.… Read more...