My phone address book is synced with iCloud and also Google Contacts.
The key is to make iPhone the primary for adding on your phone. Then occasionally load OS X Contacts and drag up from iCloud to "All Google".
For using groups on your phone, use iCloud groups.