You are correct, but it is the more efficient way of doing it given the way templates are stored. I would say the slight increase in memory usage to load in another template would be preferable to additional queries for a single template (especially if you have 15 plugin templates being loaded, would be 15 queries).
There are only really two options - cache them all, even though they may not be used, or only load them strictly when they're required, which would mean individual queries per-template. Because they're pulled from a database per-request and the cache doesn't persist in the filesystem or anything, there's limitation on how to load them.
On this forum, the query to load all templates returned 1143 rows and took 6ms - adding in another, say, 15 plugin templates here would not have any measurable effect on this query, whereas 15 new queries would.
Yes, instead of adding
all templates for a plugin, you could maybe only add them if they might be used on that page (i.e. if a template was used in the postbit, only add it to $templatelist if THIS_SCRIPT == 'showthread.php'), however if the template calls are buried deep in logic, you may still cache a template that doesn't get used on that request, and like I say, caching them all would add negligible overhead anyway. In the same way, the core files will cache every template used in that file, but there's no guarantee every one of them is going to be used, so really, plugins doing the same is the best way to go about it.