An apostrophe always takes the place of other letters. “It’s” (the contraction) always means “it is”. The possessive form of “it” is always “its”.

Yes, it matters. Yes, I knew what was intended based on context this time, but context isn’t always available or reliable. The only way to be sure you won’t be misunderstood is to follow the same rules as everyone else every time.