On the application form for Permanent Residency (
http://www.immd.gov.hk/pdforms/rop145.pdf ) you have to list any absences of more than 6 months and justify your claim that HK was still your main place of residence during that period. It states "continuous periods of 6 months" so multiple exits totalling more than 180 days wouldn't be a problem, but if you weren't here at all for a period of more than 180 days then I think you would have a hard time proving to ImmD that HK had been your place of ordinary residence for a continuous period of 7 years, which is the requirement for PR.