What on Earth is Neural Tangent Kernel?

Neural Tangent Kernel (NTK) is well known as a powerful tool that elegantly proves the convergence and generalisation of neural networks. However, as an engineering student who basically know nothing about mathematics, I find the original NTK paper a little bit opaque to understand. Driven by my intrinsic inquisitiveness (“vegetable but addicted”), I spent some time trying to figure out what on earth is NTK with every effort and managed to get an impression on that at last. This article aims to help people who are in the same boat, vegetable, but still struggling for a free ride in deep learning theory.