GWT Demystified: Generators Part 3, Meet the Oracle

In the Matrix movies, the Oracle told Neo exactly what he needed to hear. In this GWT Demystified, we'll explore TypeOracle(s) and what they need to tell Generators.

Recall from Part 2 of this series, that we created an interface called Exportable, and told GWT that whenever GWT.create(Exportable) is called, to execute our generator on the class.

Let's look at the guts of the generator code:

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.Generator;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.UnableToCompleteException;
public class ExporterGenerator extends Generator {

public String generate(TreeLogger logger, GeneratorContext ctx,
String requestedClass)
throws UnableToCompleteException {

ClassExporter classExporter = new ClassExporter(logger, ctx);
String generatedClass = classExporter.exportClass(requestedClass);
return generatedClass;

}
}

Generators by convention are not placed in the .client package, nor .public or .server, but in a package called .rebind. They are compile time Java code, not client side Javascript, nor run-time servlet code. Generators must extend Generator and override the generate(logger, context, requestedClass) method.

TreeLogger is just a logging interface for compiler diagnostics, GeneratorContext allows access to compile time state, and the requested class is the fully qualified classname of the class that was passed to GWT.create().

In my implementation, I store this call state on another class for convenience, let's take a look at my ClassExporter.

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.UnableToCompleteException;
import com.google.gwt.user.rebind.SourceWriter;
import com.google.gwt.user.rebind.ClassSourceFileComposerFactory;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.io.PrintWriter;

/**
* This class performs the generation of export methods for a single class
*
* @author Ray Cromwell <ray@timepedia.org>
*/
public class ClassExporter {
private TreeLogger logger;
private GeneratorContext ctx;
private ExportableTypeOracle xTypeOracle;
private SourceWriter sw;
private ArrayList<JExportableClassType> exported;
private HashSet<String> visited;
private static final String ARG_PREFIX = "arg";

public ClassExporter(TreeLogger logger, GeneratorContext ctx) {
this(logger, ctx, new HashSet<String>());
}

public ClassExporter(TreeLogger logger, GeneratorContext ctx,
HashSet<String> visited) {
this.logger = logger;
this.ctx = ctx;
// a type oracle that can answer questions about whether types are
// exportable
xTypeOracle = new ExportableTypeOracle(ctx.getTypeOracle(), logger);

// globally, classes that have already been exported by the current generator
this.visited = visited;

// classes exported by *this* ClassExporter instance
exported = new ArrayList<JExportableClassType>();

}

Now, this looks more meaty. Before I can start to explain this code, I need to
explain the GeneratorContext.

Interfacing to the compiler

In order to get anything done with generators, you need to access some of the compiler's internal state, and you need to be able to generate source code. The GeneratorContext entry provides the ability to get Oracles, as well as create source code files on disk.

What's an Oracle? Think of it as the compile time analog of Java Reflection or Sun's Mirror API. In particular, the TypeOracle returned by GeneratorContext.getTypeOracle() is a pretty close analog to java.lang.Reflect introspection. You can query for classes, their fields, methods, return values, parameters, superclasses, subclasses, etc.

The GeneratorContext also exposes a tryCreate() method that allows one to create Java source files on disk, giving a PrintWriter in return. It returns null if the code has already been generated (generators may be invoked more than once), which is your signal to exit early. In practice, you usually don't use tryCreate() directly, but use helper classes which I'll explain shortly.

We create a class called ExportableTypeOracle whose job it is to answer questions about Exportable types, such as which methods are exportable, which parameters are exportable, what's the JavaScript name of a particular method going to be, etc.

Creating source files

Let's look at the main function, paired down a little bit:

/**
* This method generates an implementation class that implements Exporter
* and returns the fully qualified name of the class.
*
* @param requestedClass
* @return qualified name of generated Exporter class
* @throws UnableToCompleteException
*/
public String exportClass(String requestedClass)
throws UnableToCompleteException {

// JExportableClassType is a wrapper around JClassType
// which provides only the information and logic neccessary for
// the generator
JExportableClassType requestedType =
xTypeOracle.findExportableClassType(requestedClass);

// add this so we don't try to recursively reexport ourselves later
exported.add(requestedType);
visited.add(requestedType.getQualifiedSourceName());

if (requestedType == null) {
logger.log(
TreeLogger.ERROR, "Type '"
+ requestedClass + "' does not implement Exportable", null
);
throw new UnableToCompleteException();
}

// get the name of the Java class implementing Exporter
String genName = requestedType.getExporterImplementationName();

// get the package name of the Exporter implementation
String packageName = requestedType.getPackageName();

// get a fully qualified reference to the Exporter implementation
String qualName =
requestedType.getQualifiedExporterImplementationName();


sw = getSourceWriter(
logger, ctx, packageName,
genName, "Exporter");
if (sw == null) {
return qualName; // null, already generated
}

So we look up our class in the ExportableTypeOracle. If you get null, it means the class isn't Exportable. We then ask for an implementation class name ("Foo" yields class "FooExporterImpl implements Exporter") We then ask getSourceWriter to
create such a class for us on disk and return a Writer. Let's look at that method:

/**
* Get SourceWriter for following class and preamble
* package packageName;
* import com.google.gwt.core.client.GWT;
* import org.timepedia.exporter.client.Exporter;
* public class className implements interfaceName (usually Exporter) {
*
* }
*
* @param logger
* @param context
* @param packageName
* @param className
* @param interfaceNames vararg list of interfaces
* @return
*/
protected SourceWriter getSourceWriter(
TreeLogger logger, GeneratorContext context,
String packageName, String className, String... interfaceNames) {
PrintWriter printWriter = context.tryCreate(
logger, packageName, className
);
if (printWriter == null) {
return null;
}
ClassSourceFileComposerFactory composerFactory =
new ClassSourceFileComposerFactory(packageName, className);
composerFactory.addImport("com.google.gwt.core.client.GWT");
for (String interfaceName : interfaceNames) {
composerFactory.addImplementedInterface(interfaceName);
}

composerFactory.addImport("org.timepedia.exporter.client.Exporter");
return composerFactory.createSourceWriter(context, printWriter);
}

Here we use helper classes provided by GWT to create a class with a given packageName and className. We add imports, add interfaces, and then create a SourceWriter to write to the PrintWriter. SourceWriter differs primarily from PrintWriter in that it can keep track of indent level, and indent generated code.

The rest of the exportClass method looks like this:

sw.indent();

// here we define a JSNI Javascript method called export0()
sw.println("public native void export0() /*-{");
sw.indent();

// if not defined, we create a Javascript package hierarchy
// foo.bar.baz to hold the Javascript bridge
declarePackages(requestedType);

// export Javascript constructors
exportConstructor(requestedType);

// export all static fields
exportFields(requestedType);

// export all exportable methods
exportMethods(requestedType);

// add map from TypeName to JS constructor in ExporterBase
registerTypeMap(requestedType);

sw.outdent();
sw.println("}-*/;");

sw.println();

// the Javascript constructors refer to static factory methods
// on the Exporter implementation, referenced via JSNI
// We generate them here
exportStaticFactoryConstructors(requestedType);

// finally, generate the Exporter.export() method
// which invokes recursively, via GWT.create(),
// every other Exportable type we encountered in the exported ArrayList
// ending with a call to export0()

genExportMethod(requestedType, exported);
sw.outdent();

sw.commit(logger);

// return the name of the generated Exporter implementation class
return qualName;
}

This article is too big to delve into every one of this methods, so I will explain just one: exportFields, but first, you will finally have to meet the TypeOracle.

The Oracle that tells you exactly what you need

I created a helper class called ExportableTypeOracle that answers only the questions relevent for my generator:

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.*;
import com.google.gwt.core.ext.TreeLogger;


public class ExportableTypeOracle {
private TypeOracle typeOracle;
private TreeLogger log;
static final String EXPORTER_CLASS =
"org.timepedia.exporter.client.Exporter";
static final String EXPORTABLE_CLASS =
"org.timepedia.exporter.client.Exportable";
private JClassType exportableType = null;
private JClassType jsoType = null;
private JClassType stringType = null;

public static final String GWT_EXPORT_MD = "gwt.export";
private static final String GWT_NOEXPORT_MD = "gwt.noexport";
public static final String JSO_CLASS =
"com.google.gwt.core.client.JavaScriptObject";
private static final String STRING_CLASS = "java.lang.String";
private static final String GWT_EXPORTCLOSURE = "gwt.exportClosure";

public ExportableTypeOracle(TypeOracle typeOracle, TreeLogger log) {
this.typeOracle = typeOracle;
this.log = log;
exportableType = typeOracle.findType(EXPORTABLE_CLASS);
jsoType = typeOracle.findType(JSO_CLASS);
stringType = typeOracle.findType(STRING_CLASS);
assert exportableType != null;
}

public JExportableClassType findExportableClassType(String requestedClass) {
JClassType requestedType = typeOracle.findType(requestedClass);
if (requestedType != null) {

if (requestedType.isAssignableTo(exportableType)) {
return new JExportableClassType(this, requestedType);
}
}
return null;
}

The first method needed is findExportableClassType. Given a request like "org.foo.bar.MyClass", it looks this type up in the underlying TypeOracle via findType(), and if found, it checks to see if the JClassType that is returned can be assigned to org.timepedia.exporter.client.Exportable. If so, this class is deemed Exportable, and it returns a wrapper around JClassType that can answer more questions about fields, methods, etc.

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.JClassType;
import com.google.gwt.core.ext.typeinfo.JField;
import com.google.gwt.core.ext.typeinfo.JMethod;
import com.google.gwt.core.ext.typeinfo.JConstructor;

import java.util.ArrayList;


public class JExportableClassType implements JExportable, JExportableType {
private ExportableTypeOracle exportableTypeOracle;
private JClassType type;
private static final String IMPL_EXTENSION = "ExporterImpl";

public JExportableClassType(ExportableTypeOracle exportableTypeOracle,
JClassType type) {
this.exportableTypeOracle = exportableTypeOracle;
this.type = type;
}



public JExportableField[] getExportableFields() {
ArrayList<JExportableField> exportableFields =
new ArrayList<JExportableField>();

for (JField field : type.getFields()) {
if (ExportableTypeOracle.isExportable(field)) {
exportableFields.add(new JExportableField(this, field));
}

}
return exportableFields.toArray(new JExportableField[0]);
}

For brevity, I only show the getExportableFields() method. This method enumerates over the real ClassType.getFields() method, and for each JField entry, asks the ExportableTypeOracle if it is exportable or not. If so, it creates another wrapper around JField called JExportableField and adds it to the returned results.

But what makes a field exportable?

public static boolean isExportable(JField field) {
return field.isStatic() &&
field.isPublic() &&
field.isFinal() &&
( isExportable(field.getMetaData(GWT_EXPORT_MD)) ||
( isExportable(field.getEnclosingType()) &&
!isNotExportable(
field.getMetaData(GWT_NOEXPORT_MD))));
}

A field is exportable if and only if it is: 1) "public static final", 2) it has a @gwt.export annotation OR its class has a @gwt.export annotation and the field does not have a @gwt.noexport annotation.

Finally, let's look at the exportFields() method.

/**
* For each exportable field Foo, we generate the following Javascript:
* $wnd.package.className.Foo = JSNI Reference to Foo
*
* @param requestedType
* @throws UnableToCompleteException
*/
private void exportFields(JExportableClassType requestedType)
throws UnableToCompleteException {
for (JExportableField field : requestedType.getExportableFields()) {
sw.print("$wnd." + field.getJSQualifiedExportName() + " = ");
sw.println("@" + field.getJSNIReference() + ";");
}
}

Self-explanatory except for the getJSQualifiedExportName() and getJSNIRefernece(). The former method will return the JavaScript package + JavaScript constructor + exported name of the field. So for a field "FOO" on class "org.foo.Bar", this string might be "org.foo.Bar.FOO" if none of the exported names were overriden with an annotation.

The getJSNIReference() function will return a string like "org.foo.Bar::FOO" to reference to the public static final field of that class.

Finally, when done generating, you call SourceWriter.commit(logger), and return the name of the class you generated.

That about wraps up this edition of GWT Demystified. The GWT Exporter source and module is now available on http://code.google.com/p/gwt-exporter

-Ray