Consuming partially embedded data with Ember

A service I am calling (which I have no control over) returns the following JSON payload:

{
   name: 'name',
   tags: [
       {
           name: 'name',
           users: [ 2, 3, 4 ]
       },
       // ........
   ]
}

I have written a custom serializer to handle the embedded data:

export default DS.RESTSerializer.extend(
    DS.EmbeddedRecordsMixin, {
        attrs: {
            tags: { embedded: 'always' },
        }
    }
);

Now the users within Tags is not embeded. How do I tell Ember that it needs to go fetch the entities based on the user IDs within the tags object?