How important is business insurance in an early startup?

Any insurance that is "compulsory" by state law should obviously be considered first. In most cases, this is Worker's Compensation insurance. If you have ANY non-owner employees, chances are you are required to offer some form of coverage for both their protection and yours.

Second are any coverages which may be required by your building lessor or lienholder, such as general liability / commercial property coverage. It is also worth noting that even in cases where you are not specifically required to carry this coverage, operating your business without general liability in place can make it exceedingly difficult (and expensive) to obtain this coverage at a later date.

Finally, coverage to protect the owners/principals/directors should be put in place. This brings two other coverages into play:
1) Professional liability insurance (also known as errors and ommissions or malpractice). Generally this is only needed if you are providing some sort of service for a fee. Product-based startups are less (or not at all) likely to need this.
2) Directors & officers insurance. This specifically indemnifies the company principals and board of directors (if present) from liability claims arising out of the operations of the company. This is sometimes required by VC as part of the funding agreement.

These are what I would consider the "must-have" coverages for any new business. Of secondary importance are things like employee benefits and the like, which serve to assist in company growth, acquisition, and retention but are not required to run a business.