In preparation for the total solar eclipse of 2024, the totality path crosses my hometown

I have basic knowledge of optics and according to what I know, a front filter should not shift focus distance when focusing on very distant objects. The optical distance changes by the refraction index and the thickness of the material, but since the filter is at the front, for distant objects like the Sun the shift in optical distance is neglectable.
I have read and heard from several solar imager 'experts' that you need to adjust focus when you remove the front solar filter. Is this true? If so, why?
Any feedback is welcome!
Clear skies.
José