Research notebook
Papers, derivations, experiments, and implementation notes from 2015 onward.
N / 01
43 records
Volt argues that a vanilla transformer, with the right recipe and enough data, no longer needs a convolution-specialized backbone for 3D scene understanding. We took that seriou...
3dtransformerconvolution
Imagine asking a robot: “bring me the Snoopy plush from the corner of the room.” For this to work, the robot needs to find the Snoopy in 3D space — a category it has likely neve...
3dsegmentationopen-vocabulary
Predicting 3D rotations from neural networks sounds straightforward—output a rotation matrix and move on. But rotations live on $\mathrm{SO}(3)$, a curved 3-dimensional manifold...
rotation3ddeep-learning
CuTe DSL Basics — From Hello to Tiled Kernels This tutorial turns the CuTe DSL script snippets into a connected story: we start with a first GPU kernel, learn how dynamic print...
cutlasscutecuda
CUDA Memory Load/Store Performance: A Comprehensive Benchmark Analysis GPU memory performance is often the bottleneck in high-performance computing applications. Understanding ...
cudagpu-programmingperformance-optimization
Authors Junha Lee1,2,*, Chunghyun Park1,2,*, Jaesung Choe1, Yu-Chiang Frank Wang1, Jan Kautz1, Minsu Cho2, Chris Choy1 1NVIDIA, 2POSTECH * indicates equal contribution Abstr...
3d-segmentationfoundation-modelsopen-vocabulary
Authors Wei Dong, Christopher Choy, Charles Loop, Or Litany, Yuke Zhu, Anima Anandkumar IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023 Abstract Indo...
monocular-reconstructionsdfsparse-voxel-grid
Authors Yoonwoo Jeong, Seungjoo Shin, Junha Lee, Christopher Choy, Anima Anandkumar, Minsu Cho, Jaesik Park NeurIPS, 2022 Abstract The recent progress in implicit 3D represe...
neural-radiance-fieldsperceptionnerf
Authors Yufei Wang, Zhou Xian, Feng Chen, Tsun-Hsuan Wang, Yian Wang, Katerina Fragkiadaki, Christopher Choy, Zackory Erickson, David Held RSS, 2022 Model overview from the...
roboticsmanipulationdeformable-objects
Authors Shiyi Lan, Zhiding Yu, Christopher Choy, Subhashree Radhakrishnan, Guilin Liu, Yuke Zhu, Larry S. Davis, Anima Anandkumar IEEE International Conference on Computer Vis...
weakly-supervisedinstance-segmentationsemantic-correspondence
Authors Yoonwoo Jeong, Seokjun Ahn, Christopher Choy, Animashree Anandkumar, Minsu Cho, Jaesik Park ICCV, 2021 Abstract In this work, we propose a camera self-calibration al...
neural-radiance-fieldscamera-calibration3d-reconstruction
Sparse voxel convolutions are permutation invariant because convolution depends on coordinates and random access, not contiguous storage.
neural-networkssparse-convolution
The Neural Radiance Fields (NeRF) proposed an interesting way to represent a 3D scene using an implicit network for high fidelity volumetric rendering. Compared with traditional...
neural-networksrendering3d
Setting class attributes in python can be tedious. In this post, I want to summarize a trick that I’ve been using to simplify this process. Class Attributes in init In many ca...
pythonprogramming
Documentation probably is one of the most important tasks that no one has time for. I also overlook the importance as I get swept by a series of projects and requests. Recently,...
memoryneurosciencedocumentation
Many problems in science and engineering can be formulated in terms of geometric patterns in high-dimensional spaces. We present high-dimensional convolutional networks (ConvNet...
geometric-pattern-recognitionhigh-dimensional-convolutional-networkssparse-convolution
We present Deep Global Registration, a differentiable framework for pairwise registration of real-world 3D scans. Deep global registration is based on three modules: a 6-dimens...
3d-reconstructionglobal-registration3d-features
Abstract The automation of mechanical tasks brought the modern world unprecedented prosperity and comfort. However, the majority of automated tasks have been simple mechanical ...
3d-perceptionneural-networkscomputer-vision
Authors Christopher Choy, Jaesik Park, Vladlen Koltun International Conference on Computer Vision (ICCV), 2019 Speed vs. accuracy Pareto frontier of prior methods and FCGF....
representation-learning3d-featurescomputer-vision
Pytorch is a great neural network library that has both flexibility and power. Personally, I think it is the best neural network library for prototyping (advanced) dynamic neura...
pytorchextension
In many robotics and VR/AR applications, 3D-videos are readily-available sources of input (a continuous sequence of depth images, or LIDAR scans). However, those 3D-videos are p...
spatio-temporal-perception4d-convolutional-neural-networksparse-convolution
Abstract We present a method for generating colored 3D shapes from natural language. To this end, we first learn joint embeddings of freeform text descriptions and colored 3D s...
3d-reconstructiongenerative-adversarial-networknatural-language-processing
Mathematical notation is the convention that we all use to denote a concept in a concise mathematical formulation, yet sometimes there is more than one way to express the same e...
matrix-calculus
In Machine Learning, supervised problems can be categorized into regression or classification problems. The categorization is quite intuitive as the name indicate. For instance,...
lossf-divergenceregression
We have heard enough about the great success of neural networks and how they are used in real problems. Today, I want to talk about how it was so successful (partially) from an ...
data-processing-inequalityinformation-theoryneural-networks
A Gaussian process is a non-parametric model which can represent a complex function using a growing set of data. Unlike a neural network, which can also learn a complex function...
gaussian-process
3D reconstruction from a single image is a key problem in multiple applications ranging from robotic manipulation to augmented reality. Prior methods have tackled this problem t...
3d-convolutional-neural-networkfree-form-deformation3d-reconstruction
Volumetric 3D reconstruction has witnessed a significant progress in performance through the use of deep neural network based methods that address some of the limitations of tra...
generative-adversarial-network3d-reconstruction
In the previous post, we covered variational inference and how to derive update equations. In this post, we will go over a simple Gaussian Mixture Model with the Dirichlet prior...
variational-inferenceexpectation-maximization
Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In...
scene-graphrecurrent-neural-network
We introduce a Deep Stochastic IOC1 RNN Encoder- decoder framework, DESIRE, with a conditional Variational Auto-Encoder and multiple RNNs for the task of future predictions of m...
predictionrecurrent-neural-networkinverse-reinforcement-learning
Abstract 3D semantic scene labeling is fundamental to agents operating in the real world. In particular, labeling raw 3D point sets from sensors provides fine-grained semantics...
3d-vision3d-semantic-segmentationneural-networks
Statistical inference involves finding the right model and parameters that represent the distribution of observations well. Let $\mathbf{x}$ be the observations and $\theta$ be ...
expectation-maximizationvariational-inference
Statistical inference often requires modeling the distribution of data. There are two branches of statistical modeling: parametric and non-parametric methods. The former on...
dirichlet-processnonparametric-bayesian
We present a deep learning framework for accurate visual correspondences and demonstrate its effectiveness for both geometric and semantic matching, spanning across rigid motion...
correspondenceneural-networksfully-convolutional-neural-network
Inspired by the recent success of methods that employ shape priors to achieve robust 3D reconstructions, we propose a novel recurrent neural network architecture that we call th...
multi-viewsingle-viewreconstruction
Python layer in Caffe can speed up development process Issue1703 Compile WITH_PYTHON_LAYER option First, you have to build Caffe with WITH_PYTHON_LAYER option 1. Run make clea...
Parametric Regression uses a predefined function form to fit the data best (i.e, we make an assumption about the distribution of data by implicitly modeling them as linear, quad...
gaussian-process-regression
Caffe uses Google Protocol buffer and LMDB or LevelDB to save data in a single unified database file. This allows faster data loading. Saving Database in LMDB I will not cover...
google-protocol-bufferpython
To convert a mesh into a point cloud, one has to sample points that can uniformly cover the surface. To do so, one must choose the number of samples proportional to the area of ...
meshpoint-cloudbarycentric-coordinate
Caffe is one of the most popular open-source neural network frameworks. It is modular, clean, and fast. Extending it is tricky but not as difficult as extending other frameworks...
caffe
Computing the neural network gradient requires very simple calculus, yet can be tedious. Affine Transformation (Fully Connected Layer) Gradients For a simple fully connected l...
back-propagationneural-networks
$ \DeclareMathOperator*{\argmax}{arg\,max} $ Matrix norms and singular values have special relationships. In this post, I’ll summarize a few interesting properties of...
matrix-norm