Usecase: When many if not all your Taskbar Icons are missing even though the button is still there.
The directions in the source are plenty, but the specific steps below worked for me...
METHOD #2: Delete IconCache from your computer
Deleting IconCache data base file from your computer will solve this issue.
1. Press Windows key+R to open Run window on your computer.
2. In Run window, copy-paste this run command and then hit Enter.
%appdata%
Roaming folder will be opened on your computer.
3. Now, in Roaming folder, click on “Appdata” in the address bar to go to folder.
4. In Appdata folder, double click on “Local” folder to access it.
5. In Local folder, click on “View” in the menu bar. Notice if the option “Hidden items” is checked.
Otherwise, check the option to see the hidden items in the folder.
6. In Local folder, right click on “IconCache” folder and then click on “Delete” to delete the data base file from your computer.
IconCache file will be deleted from your computer.
Close File Explorer window on your computer.
Restart your computer. After rebooting your computer icons will be visible on Task bar once again.
Your problem should be solved.
SOURCE: https://thegeekpage.com/fix-taskbar-icons-missing-windows-10/