You, your wife and kids should be fine. I'd use caution getting around late at night, but the lifestyle you outline seems both sane and safe.
Island Academy is lovely.
Being in a condo complex assures a level of security and some neighbors for company and socializing - all in all it sounds like a good plan to me.