There are many improvements in the new PyTorch framework, however the most notable change is the adoption of a Dynamic Computational Graph. There are some lesser known frameworks that have this capability (i.e. Chainer and Dynet), in fact PyTorch borrowed a lot of ideas from Chainer. This capability is also referred to as “Define by Run” as opposed to the more conventional “Define and Run”. Basically, DL frameworks maintain a computational graph that defines the order of computations that are required to be

Source: PyTorch, Dynamic Computational Graphs and Modular Deep Learning – Intuition Machine – Medium