I want to upgrade NOP version 3.9 to 4.5

But if they are already installed and configured in the 4.10 version, how should I start the upgrade?

Yes, the default plugins are all there, but they may not necessarily be "Installed".  If you are using any, then "Install" and test them.  Then, one by one, add third party plugins.

I have also third party plugins installed in the version 4.10 and a custom template.

It's best to add and test third party plugins one at a time.  

Some plugins may fail if you "Install" them from the plugin list, because they may try to create the DB tables that already exist.  So, you should contact the third party and ask for their recommended upgrade process.  (You may find that information in their documentation or a 'readme' file that comes in their download package).