Tag: Deep Learning

Fat tails are weird

Posted by Filip Piekniewski

If you have taken a statistics class it may have included stuff like basic measure theory. Lebesgue measures and integrals and their relations to other means of integration. If your course was math heavy (like mine was) it may have included Carathéodory's extension theorem and even basics of operator theory on Hilbert spaces, Fourier transforms etc. Most of this mathematical tooling would be devoted to a proof of one of the most important theorems on which most of statistics is based - central limit theorem (CLT). 

Central limit theorem states that for a broad class of what we in math call random variables (which represent realizations of some experiment which includes randomness), as long as they satisfy certain seemingly basic conditions, their average converges to a random variable of a particular type, one we call normal, or Gaussian. 

The two conditions that these variables need to satisfy are that they are:

  1. Independent
  2. Have finite variance

In human language this means that individual random measurements (experiments) "don't know" anything about each other, and that each one of these measurements "most of the time" sits within a bounded range of values, as in it can actually be pretty much always … Read more...

Ai Reflections

Posted by Filip Piekniewski

Statisticians like to insist that correlation should not be confused with causation. Most of us intuitively understand this actually not a very subtle difference. We know that correlation is in many ways weaker than causal relationship. A causal relationship invokes some mechanics, some process by which one process influences another. A mere correlation simply means that two processes just happened to exhibit some relationship, perhaps by chance, perhaps influenced by yet another unobserved process, perhaps by an entire chain of unobserved and seemingly unrelated processes. 

When we rely on correlation, we can have models that are very often correct in their predictions, but they might be correct for all the wrong reasons. This distinction between weak, statistical relationship and a lot stronger, mechanistic, direct, dynamical, causal relationship is really at the core of what in my mind is the fatal weakness in contemporary approach in AI. 

The argument

Let me role play, what I think is a distilled version of a dialog between an AI enthusiast and a skeptic like myself: 

AI enthusiast: Look at all these wonderful things we can do now using deep learning. We can recognize images, generate images, generate reasonable answers to questions, this is … Read more...

AI - the no bullshit approach

Posted by Filip Piekniewski

Intro

Since many of my posts were mostly critical and arguably somewhat cynical [1], [2], [3], at least over the last 2-3 years, I decided to switch gears a little and let my audience know I'm actually a very constructive, busy building stuff most of the time, while my ranting on the blog is mostly a side project to vent, since above everything I'm allergic to naive hype and nonsense. 

Nevertheless I've worked in the so called AI/robotics/perception for at least ten years in industry now (and prior to that having done a Phd and a rather satisfying academic career), I've had a slightly different path than many working in the same area and hence have a slightly unusual point of view. For those who never bothered to read the bio, I was excited about connectionism way before it was cool, got slightly bored by it and got drawn into more bio-realistic/neuroscience based models, worked on that a few years and got disillusioned, then worked on robotics for a few years, got disillusioned by that, went on and got a DARPA grant to build the predictive vision model (which summarized all I learned about … Read more...