• Defining Entities
  • The Basics
  • Polymorphism
    • Native Representation
  • Embedding
    • Embedded Classes
    • Embedded Collections and Arrays
    • Native Representation
  • Serializing
    • Compression
  • Relationships
    • Keys
    • Ref<?> s
    • @Parent Relationships
    • One-To-Many Relationships
• Registering Entities

Defining Entities

Entities are simple Java POJOs with a handful of special annotations. Objectify has its own annotations and does NOT use JPA or JDO annotations.

Note that throughout this documentation we will leave off getter and setter methods for brevity.

The Basics

Here is a minimal entity class:

import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Ignore;

@Entity
public class Car {
    @Id Long id;
    String vin;
    int color;
    byte[] rawData;
    @Ignore irrelevant;

    private Car() {}

    public Car(String vin, int color) {
        this.vin = vin;
        this.color = color;
    }
}

Observe:

• Entity classes must be annotated with @Entity.

• Objectify persists fields and only fields. It does not arbitrarily map fields to the datastore; if you want to change the way a field is stored, rename the field. Getters and setters are ignored so you can isolate the public interface of your class (eg, public String getVehicleIdentificationNumber() { return vin; }).

• Objectify will not persist static fields, final fields, or fields annotated with @Ignore. It will persist fields with the transient keyword, which only affects serialization.

• Entities must have one field annotated with @Id. The actual name of the field is irrelevant and can be renamed at any time, even after data is persisted. This value (along with the kind ‘Car’) becomes part of the Key which identifies an entity.

• The @Id field can be of type Long, long, or String. If you use Long and save an entity with a null id, a numeric value will be generated for you using the standard GAE allocator for this kind. If you use String or the primitive long type, values will never be autogenerated.

• You can persist any of the core value types, Collections (ie Lists and Sets) of the core value types, or arrays of the core value types. You can also persist properties of type Key (the native datastore type) and Key<?>) (the Objectify generified version). There are more advanced types that can be persisted which will be discussed later.

• There must be a no-arg constructor (or no constructors – Java creates a default no-arg constructor). The no-arg constructor can have any protection level (private, public, etc).

• String fields which store more than 500 characters (the GAE limit) are automatically converted to Text internally. Note that Text fields, likeBlob fields, are never indexed (see Queries).

• byte[] fields are automatically converted to Blob internally. However, Byte[] is persisted “normally” as an array of (potentially indexed)Byte objects. Note that GAE natively stores all integer values as a 64-bit long.

Polymorphism

Objectify lets you define a polymorphic hierarchy of related entity classes, and then load and query them without knowing the specific subtype. Here are some examples:

import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.EntitySubclass;
import com.googlecode.objectify.annotation.Id;

@Entity
public class Animal {
    @Id Long id;
    String name;
}

@EntitySubclass(index=true)
public class Mammal extends Animal {
    boolean longHair;
}

@EntitySubclass(index=true)
public class Cat extends Mammal {
    boolean hypoallergenic;
}

Things to note:

• The root of your polymorphic hierarchy must be annotated with @Entity.
• All polymorphic subclasses must be annotated with @EntitySubclass.
• You can skip @EntitySubclass on intermediate classes which will never be materialized or queried for.
• You should register all classes in the hierarchy separately, but order is not important.
• Polymorphism applies only to entities, not to @Embed classes.

In a polymorphic hierarchy, you can load() without knowing the actual type:

Animal annie = new Animal();
annie.name = "Annie";
ofy().save().entity(annie).now();

Mammal mam = new Mammal();
mam.name = "Mam";
m.longHair = true;
ofy().save().entity(mam).now();

Cat nyan = new Cat();
nyan.name = "Nyan";
nyan.longHair = true;
nyan.hypoallergenic = true;
ofy().save().entity(nyan).now();

// This will return the Cat
Animal fetched = ofy().load().type(Animal.class).id(nyan.id).get();

// This query will produce three objects, the Animal, Mammal, and Cat
Query<Animal> all = ofy().load().type(Animal.class);

// This query will produce the Mammal and Cat
Query<Mammal> mammals = ofy().load().type(Mammal.class);

Native Representation

When you store a polymorphic entity subclass your entity is stored with two additional, hidden synthetic properties:

• ^d holds a discriminator value for the concrete class type. This defaults to the class shortname but can be modified with the@EntitySubclass(name="alternate") annotation.
• ^i holds an indexed list of all the discriminators relavant to a class; for example a Cat would have [[“Mammal”, “Cat]]. Note that subclasses are not indexed by default. See below.

The indexed property is what allows polymorphic queries to work. It also means that you cannot simply change your hierarchy arbitrarily and expect queries to continue to work as expected – you may need to re-save() all affected entities to rewrite the indexed field.

There are two ways you can affect this:

1. You can control indexing of subclasses by specifying @EntitySubclass(index=true). Subclasses are not indexed by default, you must explicitly enable this for each subclass. Note that this only affects queries; you can always get-by-key or get-by-id and receive the proper typed object irrespective of indexing.
2. You can use @EntitySubclass(alsoLoad="OldDiscriminator") to “reclaim” old discriminator values when changing class names. Note that this will not help with query indexes, which must be re-saved().

Embedding

Objectify supports embedded classes and collections of embedded classes. This allows you to store structured data within a single POJO entity in a way that remains queryable. With a few limitations, this can be an excellent replacement for storing JSON data.

Embedded Classes

You can nest objects to any arbitrary level.

import com.googlecode.objectify.annotation.Embed;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;

@Embed
class LevelTwo {
    String bar;
}

@Embed
class LevelOne {
    String foo;
    LevelTwo two
}

@Entity
class EntityWithEmbedded {
    @Id Long id;
    LevelOne one;
}

Embedded Collections and Arrays

You can place @Embed objects in collections and arrays:

@Entity
class EntityWithEmbeddedCollection {
    @Id Long id;
    List<LevelOne> ones = new ArrayList<LevelOne>();
}

Some things to keep in mind:

• This does not support two-dimensional structures of any kind.
  • You cannot nest @Embed arrays/collections inside other @Embed arrays/collections.
  • You cannot put arrays/collections of native types inside @Embed arrays/collections.
• You can, however, nest @Embed arrays/collections inside any number of @Embed classes.
• You should initialize collections. Null or empty collections are not written to the datastore and therefore get ignored during load. Furthermore, the concrete instance will be used as-is, allowing you to initialize collections with Comparators or other state.

Native Representation

You may wish to know how @Embed classes are persisted so that you an access them through the Low-Level API. Here is an example, given the example data structures specified above.

EntityWithEmbedded ent = new EntityWithEmbedded();
ent.one = new LevelOne();
ent.one.foo = "Foo Value";
ent.one.two = new LevelTwo();
ent.one.two.bar = "Bar Value";

ofy().save().entity(ent);

This will produce an entity that contains:

one.foo	“Foo Value”
one.two.bar	“Bar Value”

For collections and arrays of @Embed classes, the storage mechanism is more complicated:

EntityWithEmbeddedCollection ent = new EntityWithEmbeddedCollection();
for (int i=1; i<=4; i++) {
    LevelOne one = new LevelOne();
    one.foo = "foo" + i;
    one.two = new LevelTwo();
    one.two.bar = "bar" + i;

    ent.ones.add(one);
}

ofy().save().entity(ent);

This will produce an entity that contains:

ones.foo	["foo1", "foo2", "foo3", "foo4"]
ones.two.bar	["bar1", "bar2", "bar3", "bar4"]

This is what the entity would look like if the second and third values in the ones collection were null:

ones.foo^null	[1, 2]
ones.foo	["foo1", "foo4"]
ones.two.bar	["bar1", "bar4"]

The synthetic ^null property only exists if the collection contains nulls. It is never indexed.

Serializing

An alternative to @Embed is to @Serialize, which will let you store nearly any Java object graph.

@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize Map<Object, Object> stuff;
}

There are some limitations:

• All objects stored in the graph must follow Java serialization rules, including implement java.io.Serializable.
• The total size of an entity cannot exceed 1 megabyte. If your serialized data exceeds this size, you will get an exception when you try to save() it.
• You will not be able to use the field or any child fields in queries.
• As per serialization rules, transient (the java keyword) fields will not be stored.
• All Objectify annotations will be ignored within your serialized data structure. This means @Ignore fields within your serialized structure will be stored!
• Java serialization data is opaque to the datastore viewer and other languages (ie GAE/Python). You will only be able to retrieve your data from Java.

However, there are significant benefits to storing data this way:

• You can store nearly any object graph – nested collections, circular object references, etc. If Java can serialize it, you can store it.
• Your field need not be statically typed. Declare Object if you want.
• Collections can be stored in their full state; for example, a SortedSet will remember its Comparator implementation.
• @Serializ collections can be nested inside @Embed collections.

You are strongly advised to place serialVersionUID on all classes that you intend to store as @Serialize. Without this, any change to your classes will prevent stored objects from being deserialized on fetch. Example:

class SomeStuff implements Serializable {
    /** start with 1 for all classes */
    private static final long serialVersionUID = 1L;

    String foo;
    Object bar;
}

@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize SomeStuff stuff;
}

NOTA:

The serialization runtime associates with each serializable class a version number, called a serialVersionUID, which is used during deserialization to verify that the sender and receiver of a serialized object have loaded classes for that object that are compatible with respect to serialization. If the receiver has loaded a class for the object that has a different serialVersionUID than that of the corresponding sender’s class, then deserialization will result in an InvalidClassException. A serializable class can declare its own serialVersionUID explicitly by declaring a field named “serialVersionUID” that must be static, final, and of type long:

ANY-ACCESS-MODIFIER static final long serialVersionUID = 42L;

If a serializable class does not explicitly declare a serialVersionUID, then the serialization runtime will calculate a default serialVersionUID value for that class based on various aspects of the class, as described in the Java(TM) Object Serialization Specification. However, it is strongly recommended that all serializable classes explicitly declare serialVersionUID values, since the default serialVersionUID computation is highly sensitive to class details that may vary depending on compiler implementations, and can thus result in unexpected InvalidClassExceptions during deserialization. Therefore, to guarantee a consistent serialVersionUID value across different java compiler implementations, a serializable class must declare an explicit serialVersionUID value. It is also strongly advised that explicit serialVersionUID declarations use the private modifier where possible, since such declarations apply only to the immediately declaring class–serialVersionUID fields are not useful as inherited members.

 

Compression

You can automatically zip the serialized blob:

@Entity
class EntityWithSerialized {
    @Id Long id;
    @Serialize(zip=true) Map<Object, Object> stuff;
}

You can add or remove zipping at any time. Irrespective of the zip value, Objectify will read the field correctly whether it was zipped or not in the datastore. Of course, when the entity is saved, the zip value will determine whether or not the data in that instance is compressed.

Relationships

A relationship is simply a key stored as a field in an entity. Objectify provides several facilities to make it easier to manage relationships using keys.

Keys

There are two different Key classes in Objectify:

1. com.google.appengine.api.datastore.Key, usually referred to as Key
2. com.googlecode.objectify.Key, usually referred to as Key<?>

#1 is the native datastore Key class used by the low-level API. It is the “one true Key” because all datastore operations use this type; Objectify always converts to (or from) the native Key. However, because this structure lacks generic type information, it should almost never appear in your code.

#2 is Objectify’s generified version of Key. It’s much less errorprone to use Key<Person> or Key<Car>, so you should always use Key<?> when you use keys directly.

For all intents and purposes, these structures are equivalent. For example, these two classes are identical in the datastore:

import com.google.appengine.api.datastore.Key;

@Entity
class Thing {
    @Id Long id;
    Key other;
}
import com.googlecode.objectify.Key;

@Entity
class Thing {
    @Id Long id;
    Key<Other> other;
}

You can change the type of the other field at any time, even after data is stored. Remember, the datastore contains the native Key.

Ref<?>s

Even Key<?>s are not very convenient when you are working with graphs of entities. Objectify provides Ref<?>, which works just like a Key<?> but can hold a reference to the actual entity object as well:

import com.googlecode.objectify.Ref;

@Entity
class Car {
    @Id Long id;
    Ref<Person> driver;    // Person is an @Entity
}

Car car = new Car();
car.driver = Ref.create(driverKey);
ofy().save().entity(car).now();

Car fetched = ofy().load().entity(car).get();
ofy().load().ref(fetched.driver);
Person driver = fetched.driver.get();

Ref<?> fields are stored as native Key fields in the datastore. You can freely swap between Key, Key<?>, and Ref<?> in your Java data model without modifying stored data.

This provides a nice consistent structure for your object graph, but it’s still clunky. Ref<?> works best in concert with the @Load annotation.

@Load

import com.googlecode.objectify.annotation.Load;

@Entity
class Car {
    @Id Long id;
    @Load Ref<Person> driver;    // Person is an @Entity
}

Car car = new Car();
car.driver = Ref.create(driverKey);
ofy().save().entity(car).now();

Car fetched = ofy().load().entity(car).get();
Person driver = fetched.driver.get();

Objectify will translate this into an optimal number of batch fetches. For example:

ofy().load().keys(car1, car2, car3);

This will produce one round of batch fetching for the cars, plus one round of batch fetching for all the drivers, plus one round of fetching for any@Load fields in all the drivers… and on. Objectify is also smart about not re-fetching anything that has already been fetched in an earlier cycle. This optimized behavior is very hard to write by hand.

Often you will not wish to @Load every entity relationship every time. @Load provides the ability to specify conditional load groups. See <insert reference to load group section> for more.

Hiding Ref<?>

You will generally find it convenient to encapsulate Ref<?> behind getter and setter methods:

@Entity
class Car {
    @Id Long id;
    @Load Ref<Person> driver;    // Person is an @Entity

    public Person getDriver() { return driver.get(); }
    public void setDriver(Person value) { driver = Ref.create(value); }
}

@Parent Relationships

Parent relationships are described in Concepts. Objectify models them with the @Parent annotation:

import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Id;
import com.googlecode.objectify.annotation.Parent;

@Entity
public class Person {
    @Id Long id;
    String name;
}

@Entity
public class Car {
    @Parent Key<Person> owner;
    @Id Long id;
    String color;
}

Each Car entity is part of the parent owner’s entity group. Important things to note:

• There can only be one @Parent field in a class!
• Along with the @Id and kind, the @Parent field defines the key (identity) of the entity.
  • Construct a key to the car like this: Key.create(ownerKey, Car.class, carId)
• The name of the field is irrelevant and can be changed at any time without modifying stored data.
• The @Parent field can be Key, Key<?>, or Ref<?>.
• @Load annotations work with Ref<?>s. As a bonus, the parent will be loaded in the same batch as the initial load.

Note that this example is an inappropriate use of the @Parent entity; if a car were to be sold to a new owner, you would need to delete the Car and create a new one. It is often better to use simple nonparent key references even when there is a conceptual parent-child or owner-object relationship; in that case you could simply change the owner.

VERY IMPORTANT: If you load() an entity, change the @Parent field, and save() the entity, you will create a new entity. The old entity (with the old parent) will still exist. You cannot simply change the value of a @Parent field. This is a fundamental aspect of the appengine datastore; @Parent values form part of an entity’s identity.

One-To-Many Relationships

The datastore can natively persist collections of simple types, including keys. This creates one approach for defining one-to-many (and many-to-many) relationships.

@Entity
public class Employee {
    @Id String name;
    List<Key<Employee>> subordinates = new ArrayList<Key<Employee>>();
}

This works with Keys, Key<?>s, and Ref<?>s. You can use @Load annotations with Ref<?>s.

There are some considerations:

• Every time you load() or save() an Employee, it will fetch and store the entire list of subordinates keys. Large numbers of subordinates will bloat the entity and impact performance.
• App Engine limits you to 5,000 entries in a multi-valued property.
• Because fetches are performed by key, you have a strongly-consistent view of the a manager and subordinates.

If you put @Index on subordinates, you can issue a query like this:

Iterable<Employee> managers = ofy().load().type(Employee.class).filter("subordinates", fred);

One-To-Many Alternative

Storing keys in a collection field is a very “appenginey” way of modeling data, and faithfully works with the key/value nature of the datastore. However, this is not the traditional way of modeling one-to-many relationships in a relational database. You can still use the RDBMS approach, but there are caveats:

@Entity
public class Employee {
    @Id String name;
    @Index Key<Employee> manager;
}

Iterable<Employee> subordinates = ofy().load().type(Employee.class).filter("manager", fred);

• This is a query and therefore weakly consistent.
• You cannot use this query inside of a transaction.
• Queries are less cache-friendly than direct key references.

The decision of which approach to take will depend heavily on the shape of your data and what queries you need to perform.

Registering Entities

Before you use Objectify to load or save data, you must register all entity classes for your application. Objectify will introspect these classes and their annotations to build a metamodel which is used to efficiently manipulate entities at runtime.

ObjectifyService.register(Car.class);
ObjectifyService.register(Motorcycle.class);

• Registration must be done at application startup, before Objectify is used.
• Registration must be single-threaded. Do not register() from multiple threads.
• All entity classes must be registered, including polymorphic subclasses.
• @Embed classes do not need to be registered.

We recommend that you register classes in a static initializer for something that is used at application startup. For example:

public class MyStartupServlet {
    static {
        ObjectifyService.register(Car.class);
        ObjectifyService.register(Motorcycle.class);
    }

    ...the rest of your servlet
}

Because this registration process requires loading and introspecting classes, it adds a small delay to your application start time which is directly proportional to the number of classes you register. While Objectify tries to minimize this delay by eschewing classpath-scanning, the delay is not entirely avoidable (classloading on appengine is very slow). The best that can be said is that this delay is significantly shorter than the equivalent JDO/JPA delay.