Posts

Day 8 - Computing Matter (Tobi Delbruck, Melika Payvand, Walter Senn)

Image
  Today felt like a shift in perspective. The discussions kept circling a deeper question: What is the (biological and artificial) substrate of computation and how much does it matter? Three speakers approached this from different angles: Tobi Delbruck → sparsity and physical limits Melika Payvand → richer neuron models and input-dependent dynamics Walter Senn → dendrites, gain modulation, and links to attention/transformers The morning discussions moved across levels: from hardware constraints (sparsity), to neural (gated) dynamics, to cognitive function (attention in dendrites), and back again to... how do we design intelligent systems? Sparsity dominates everything (?) Tobi started from a deceptively simple question: What is the actual operating regime of the brain? He sparked discussion on brain insights and shared numbers/observations to understand efficiency.   ~10¹⁵ synapses Avg ~1–10 Hz firing rates (very low) ~10⁻¹⁴ J per synaptic event (energy per synaptic operati...

Day 7 - The Real World (Charlotte Frenkel, Jörg Conradt, Guido de Croon)

Image
The Real World (Charlotte Frenkel, Jörg Conradt, Guido de Croon) After a (somewhat) relaxing Sunday, the Capocaccia workshop gets back on track with an engaging discussion about applying our insights to the dreaded "real" world. From staring into the biological and theoretical abyss, we are now talking about pragmatic applications that work. A Wishlist  Our first discussion leader, Charlotte Frenkel, kicked things off by highlighting that deep networks waste a lot of energy performing the exact same computations for every single image or token. The holy grail is data-dependent computation, in the words of Charlotte, letting "data shape your computation". To get there, Charlotte presented her wishlist for an algorithm that would map well to real-world hardware: Local in space and time : Relying on backpropagation through time is highly inefficient because it lacks locality; computations should rely strictly on information available *right now*.  Exploiting sparsity (...

Day 6 - Workgroup Updates, Industrial Experience Panel (Andre van Schaik, Tobi Delbruck, Eric Ryu, Christoph Posch, Christian Mayr), Ethics (Sara Solla) and Social Dinner

Image
The evening before, long after the last sessions had ended, the workshop “disco” was still full of activity. Some groups were debugging hardware, others were still coding, discussing, or trying to get some more results working before the morning updates. Laptops stayed open for at least a few groups until around 2 AM. A few hours later, somehow, everyone made it back for breakfast and the morning session under another day of clear skies and bright sun. The morning began with the workgroup updates. After several days of discussions, experiments, and some late evening/night-work sessions, the different project groups now gave a short overview of where they currently stood. What had worked so far, what had not, what had changed since the original project pitches, and what they planned to focus on during the second week. Some groups presented first prototypes or simulation results, software pipelines, experimental setups, or revised research directions that had emerged during the week....

Day 4 - From Neural Populations to Societies (Sara Solla, Byron Yu)

Image
  After Day 3 left us questioning whether brains should be built or grown , we stepped away from “LALA LAND” (see day 3) into something more grounded, but no less mysterious…Manifolds🍥 What does the biological brain actually do at the population level? How does collective neural activity give rise to behavior, and (how) can we use it? These were the guiding core questions throughout this morning. Sara Solla opened by reminding us how neuroscience began: single neurons, single responses, tuning curves etc.   But the brain is not a collection of isolated units. It is a system where function emerges from interactions (and at multiple levels). So the old picture that neuroscientists had in the past was incomplete to the point of even misleading.   The key shift over the past decades has been technological in modern neuroscience. From recording a handful of neurons to hundreds, thousands, even tens of thousands with tools like Neuropixels.   And what emerges from these...