The NVIDIA CUDA Deep Neural Network library (cuDNN) is a GPU-accelerated
library of primitives for deep neural networks. cuDNN provides highly tuned
implementations for standard routines such as forward and backward convolution,
pooling, normalization, and activation layers. cuDNN is part of the NVIDIA
Deep Learning SDK.

You will need to register for NVIDIA developer account to download the source.

Make sure you create the CUDA_HOME environment variable and add it to your
~/.bashrc. The default should be

CUDA_HOME=/usr/share/cuda