No one cares.
No. The people around you don’t care, especially if the following is true:
- You show up when they need you
- Your contributions matter
- You bring ideas to the table
If you are reliable, interested, and curious, people trust you. So when you have to disappear to get something done, they aren’t thinking about you. They know you bring the goods.
If you earn trust, feel free to use it.
The only thing you can’t stop doing are the three things above.