The physical presence is known as a "nexus." Each state defines nexus differently, but all agree that if you have a store or office of some sort, a nexus exists. If you are uncertain whether or not your business qualifies as a physical presence, contact your state's revenue agency. If you do not have a physical presence in a state, you are not required to collect sales taxes from customers in that state.
Here’s A Guide to Sales Tax Nexus in All U.S. States