A simple way to understand the difference between a GPU and a CPU is to compare how they process tasks. A CPU consists of a few cores optimized for sequential serial processing while a GPU has a massively parallel architecture consisting of thousands of smaller, more efficient cores designed for handling multiple tasks simultaneously.GPU does mathematically heavy tasks that usually would strain the CPU .
CUDA programming was invented to get great software to GPU , so we could run C,C++ or fotron code directly on GPU’s. NVIDIA , went out to build CUDA programming so developers can take advantage of this hardware for various tasks that strain the CPU. NVIDIA main was to , Bring Software and Hardware together. Now , we use GPU to perform any tasks that burden the CPU for example , floating point number operations. NVIDIA made it possible to harness the GPU power for its true potential and by removing the learning curve required before , to write code for GPU.
So by using CUDA extensions , we can write Kernel code in C
source : https://blogs.nvidia.com/blog/2012/09/10/what-is-cuda-2/
CUDA is not API or a language for that matter , It is just a platform to write seamless GPU code. In the above picture , you can see with simple keywords let the developer write CUDA code.