If you are looking to migrate your local SQL DB to Azure SQL see the following MS article:https://docs.microsoft.com/en-us/azure/sql-database/sql-database-migrate-your-sql-server-database
You would need to update your existing connection string to the Azure connection string in the App_Data\dataSettings.json file. If it makes it easier to duplicate the local SQL connection that exists and comment it out so you have it for your local dev.
When you copy the connection string from Azure it will start with "Server" which will need to be changed back to "Data Source". Also in the new connection string, change the "MultipleActiveResultSets" to True or you will end up with an exception.
Also you may want to look at the appsettings.json file for configurations that pertain to your particular uses.