💭 My experience as an Applied Scientist Intern at Amazon
Published:
Posts by Tags
Amazon
Applied Scientist
Backpropagation
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Bayesian Inference
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Bayesian Neural Networks
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Central Limit Theorem
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Deep Information Propagation
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Deep Neural Networks
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Fisher Information
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Gaussian Processes
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Gradient Descent
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
Industry
Inference Learning
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Infinite Width Limit
♾️ Infinite Widths Part I: Neural Networks as Gaussian Processes
Published:
This is the first post of a short series on the infinite-width limits of deep neural networks (DNNs). We start by reviewing the correspondence between neural networks and Gaussian Processes (GPs).
Internship
Interpretability
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
KAN
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Kolmogorov-Arnold Networks
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Kolmogorov-Arnold Representation Theorem
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Local Learning
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Loss Landscape
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
Machine Learning
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
Multi-layer Perceptrons
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Natural Gradient Descent
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
Neural Scaling Laws
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Normal Computing
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
PhD
Predictive Coding
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Saddle Points
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.
Saddles
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Second-Order
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Second-order Methods
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
Splines
KANs Made Simple
Published:
🤔 Confused about the recent KAN: Kolmogorov-Arnold Networks? I was too, so here’s a minimal explanation that makes it easy to see the difference between KANs and multi-layer perceptrons (MLPs).
Thermodynamic AI
💥 Thermodynamic Natural Gradient Descent
Published:
I recently came across this paper Thermodynamic Natural Gradient Descent by Normal Computing. I found it very interesting, so below is my brief take on it.
Trust Region
🧠 Predictive Coding as a 2nd-Order Method
Published:
📖 TL;DR: Predictive coding implicitly performs a 2nd-order weight update via 1st-order (gradient) updates on neurons that in some cases allow it to converge faster than backpropagation with standard stochastic gradient descent.
Vanishing Gradients
⛰️ The Energy Landscape of Predictive Coding Networks
Published:
📖 TL;DR: Predictive coding makes the loss landscape of feedforward neural networks more benign and robust to vanishing gradients.