You can drink the tap water in most places in Iceland without boiling it (especially in Reykjavik), but if you're further into the countryside then I would check first!

Love this track, can't stop listening

Relevant Scott Manley video:

https://youtu.be/cxNJoaBLLNM