Dynamically determine existence of template only Ember glimmer component?

I know that to dynamically determine the existence of a component on Ember we can use the solutions explained here or here.

We have a helper that uses .hasRegistration('component:${component}')

This also works for Glimmer components as long as there is a .js file defined, but it doesn't work for template-only Glimmer components. The component doesn't seem to be registered in that case.

Does anyone know a solution that would work for template-only glimmer components too?