PVM is out

Posted by Filip Piekniewski

So finally after many months we can share our progress. Predictive Vision Model (PVM) is a new recurrent learning architecture we've been exploring for a while now. The paper showing initial results is available here https://arxiv.org/abs/1607.06854 and the corresponding code is https://github.com/braincorp/PVM .

So what is PVM? It is a new approach to learning foundations of perception in an unsupervised way. We exploit the idea of multi-scale and multi-level stacked predictive encoders (similar to autoencoder but tries to predict the next frame in a sequence of inputs). We then find, that if we train this architecture online, we can liberally wire it with feedback and lateral connectivity and nothing breaks! So we end up with a scalable, unsupervised architecture that naturally operates in time and is able to exploit all the regularities, which are so obvious to us - humans  highly visual animals - that we don't even notice them consciously until we are faced with an optical illusion.

This is really just the beginning of the work. We experimented a lot, therefore we decided not to invest into a GPU implementations, but now this certainly is a good avenue to pursue. Recurrent feedback and online operation make it difficult … Read more...

Deep nets and the brain

Posted by Filip Piekniewski

The Deep Nets are the hot thing these days in machine learning research. So hot that institutes are being established to study the social consequences of AI overtaking humanity and the White House has concerns regarding AI. Now every respecting sceptic should ask a question: is humanity really that close to solving the secret of intelligence? Or maybe this is just hype like in the 50'ies and 80'ies?

This is a long discussion. I will post many articles on that in the future hopefully. Here lets dissect a few popular myths:

  1. Convolutional deep nets solve perception. It is true that these systems have won ImageNet by a substantial margin and often can classify the content of the image accurately. It is also known that they get fooled by stuff that certainly would not fool a human. So that indicates that there is something missing. I think that we have somewhat shallow understanding of what perception really is. Vision is not about just categorising what we see. In fact we more often than not ignore the class of what we see. Humans or animals are more interested with affordances, namely "can I perform an action on what I
… Read more...

Sysadmin hobby

Posted by Filip Piekniewski

I've recently moved from a managed hosting company to pure, raw Amazon instance and I have to say it's fun. I've set up all the LAMP stack, email server, ssl/certs and a few other services and it feels good. Of course within the first maybe 2 months my machine got attacked by a DDOS targeting WordPress installations and essentially went down.

It took me a while, one evening and one morning of stressful rebooting and fiddling to finally figure out what is going on. In the meanwhile I've mastered every aspect of running an AWS instance like detaching/attaching volumes, changing volume type, changing instance type, reassigning elastic ip's you name it. The thing manifested itself in a substantial (like 99%) IOWAIT time on the machine CPU (top -> wa) and resulting sluggishness of response. So if you are apparently running out of IO on your server, it likely is not without some shady reason - amazon limits for IO are well within what should be enough for a small server with a few websites.

Finally I looked up my apache logs (which I should have done in the first place) to see that I'm being bombarded by large post requests … Read more...

What is difficult and what is easy?

Posted by Filip Piekniewski

Recently the world has been thrilled by the game of go played between the world top player and a computer program. The program eventually won 4 of the 5 rounds, marking the historical moment in which go had finally been solved. This is almost twenty years after another important game - chess had suffered similar defeat. Why did it take almost 20 years?

You will hear that go is apparently a lot more difficult than chess and therefore the search space is much larger and bla bla bla. Well did we know in advance it was so much harder? Probably not until we started trying to solve it. Do we frankly even now have any reasonable intuition1 as to why go is so much harder? I doubt.

OK, let's look at something simpler - graph problems. Some of them are easy, lets say minimum-spanning-tree or even all-pair-shortest-paths. Some of them are extremely hard e.g. traveling salesman problem. Even though these problems sound very similar. It's all about some minimal path in a graph, it would seem they should be similar, yet their solutions vary greatly in complexity.

Similarly with machine learning problems. We are aware that some … Read more...

Introduction

Posted by Filip Piekniewski

Hello Internet,

Let me start this blog with a short introduction. My name is Filip Piekniewski and I'm a researcher working on topics of artificial intelligence, machine learning, perception etc (check my website for more info). For the past six+ years I've been working at Brain Corporation in San Diego. The company has ambitious goal of building brains for robots and the work we've been doing is quite unique. I'd like to use this blog to share some of my thoughts on Machine Learning, from a slightly different perspective than a lot of the mainstream, namely from the perspective of actually applying these techniques to a physical device existing in physical reality. As we've learned, this is a whole different ballpark than running your algorithm on a dataset in a sterile, digital world. I hope you will find this read entertaining.

 … Read more...