Saturday, June 9, 2007

GWT Demystified: Generators Part 3, Meet the Oracle

In the Matrix movies, the Oracle told Neo exactly what he needed to hear. In this GWT Demystified, we'll explore TypeOracle(s) and what they need to tell Generators.

Recall from Part 2 of this series, that we created an interface called Exportable, and told GWT that whenever GWT.create(Exportable) is called, to execute our generator on the class.

Let's look at the guts of the generator code:


package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.Generator;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.UnableToCompleteException;
public class ExporterGenerator extends Generator {

public String generate(TreeLogger logger, GeneratorContext ctx,
String requestedClass)
throws UnableToCompleteException {

ClassExporter classExporter = new ClassExporter(logger, ctx);
String generatedClass = classExporter.exportClass(requestedClass);
return generatedClass;

}
}


Generators by convention are not placed in the .client package, nor .public or .server, but in a package called .rebind. They are compile time Java code, not client side Javascript, nor run-time servlet code. Generators must extend Generator and override the generate(logger, context, requestedClass) method.

TreeLogger is just a logging interface for compiler diagnostics, GeneratorContext allows access to compile time state, and the requested class is the fully qualified classname of the class that was passed to GWT.create().

In my implementation, I store this call state on another class for convenience, let's take a look at my ClassExporter.


package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.TreeLogger;
import com.google.gwt.core.ext.GeneratorContext;
import com.google.gwt.core.ext.UnableToCompleteException;
import com.google.gwt.user.rebind.SourceWriter;
import com.google.gwt.user.rebind.ClassSourceFileComposerFactory;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.HashSet;
import java.io.PrintWriter;

/**
* This class performs the generation of export methods for a single class
*
* @author Ray Cromwell <ray@timepedia.org>
*/
public class ClassExporter {
private TreeLogger logger;
private GeneratorContext ctx;
private ExportableTypeOracle xTypeOracle;
private SourceWriter sw;
private ArrayList<JExportableClassType> exported;
private HashSet<String> visited;
private static final String ARG_PREFIX = "arg";

public ClassExporter(TreeLogger logger, GeneratorContext ctx) {
this(logger, ctx, new HashSet<String>());
}

public ClassExporter(TreeLogger logger, GeneratorContext ctx,
HashSet<String> visited) {
this.logger = logger;
this.ctx = ctx;
// a type oracle that can answer questions about whether types are
// exportable
xTypeOracle = new ExportableTypeOracle(ctx.getTypeOracle(), logger);

// globally, classes that have already been exported by the current generator
this.visited = visited;

// classes exported by *this* ClassExporter instance
exported = new ArrayList<JExportableClassType>();

}


Now, this looks more meaty. Before I can start to explain this code, I need to
explain the GeneratorContext.

Interfacing to the compiler


In order to get anything done with generators, you need to access some of the compiler's internal state, and you need to be able to generate source code. The GeneratorContext entry provides the ability to get Oracles, as well as create source code files on disk.

What's an Oracle? Think of it as the compile time analog of Java Reflection or Sun's Mirror API. In particular, the TypeOracle returned by GeneratorContext.getTypeOracle() is a pretty close analog to java.lang.Reflect introspection. You can query for classes, their fields, methods, return values, parameters, superclasses, subclasses, etc.

The GeneratorContext also exposes a tryCreate() method that allows one to create Java source files on disk, giving a PrintWriter in return. It returns null if the code has already been generated (generators may be invoked more than once), which is your signal to exit early. In practice, you usually don't use tryCreate() directly, but use helper classes which I'll explain shortly.

We create a class called ExportableTypeOracle whose job it is to answer questions about Exportable types, such as which methods are exportable, which parameters are exportable, what's the JavaScript name of a particular method going to be, etc.

Creating source files


Let's look at the main function, paired down a little bit:

/**
* This method generates an implementation class that implements Exporter
* and returns the fully qualified name of the class.
*
* @param requestedClass
* @return qualified name of generated Exporter class
* @throws UnableToCompleteException
*/
public String exportClass(String requestedClass)
throws UnableToCompleteException {

// JExportableClassType is a wrapper around JClassType
// which provides only the information and logic neccessary for
// the generator
JExportableClassType requestedType =
xTypeOracle.findExportableClassType(requestedClass);

// add this so we don't try to recursively reexport ourselves later
exported.add(requestedType);
visited.add(requestedType.getQualifiedSourceName());

if (requestedType == null) {
logger.log(
TreeLogger.ERROR, "Type '"
+ requestedClass + "' does not implement Exportable", null
);
throw new UnableToCompleteException();
}

// get the name of the Java class implementing Exporter
String genName = requestedType.getExporterImplementationName();

// get the package name of the Exporter implementation
String packageName = requestedType.getPackageName();

// get a fully qualified reference to the Exporter implementation
String qualName =
requestedType.getQualifiedExporterImplementationName();


sw = getSourceWriter(
logger, ctx, packageName,
genName, "Exporter");
if (sw == null) {
return qualName; // null, already generated
}


So we look up our class in the ExportableTypeOracle. If you get null, it means the class isn't Exportable. We then ask for an implementation class name ("Foo" yields class "FooExporterImpl implements Exporter") We then ask getSourceWriter to
create such a class for us on disk and return a Writer. Let's look at that method:

/**
* Get SourceWriter for following class and preamble
* package packageName;
* import com.google.gwt.core.client.GWT;
* import org.timepedia.exporter.client.Exporter;
* public class className implements interfaceName (usually Exporter) {
*
* }
*
* @param logger
* @param context
* @param packageName
* @param className
* @param interfaceNames vararg list of interfaces
* @return
*/
protected SourceWriter getSourceWriter(
TreeLogger logger, GeneratorContext context,
String packageName, String className, String... interfaceNames) {
PrintWriter printWriter = context.tryCreate(
logger, packageName, className
);
if (printWriter == null) {
return null;
}
ClassSourceFileComposerFactory composerFactory =
new ClassSourceFileComposerFactory(packageName, className);
composerFactory.addImport("com.google.gwt.core.client.GWT");
for (String interfaceName : interfaceNames) {
composerFactory.addImplementedInterface(interfaceName);
}

composerFactory.addImport("org.timepedia.exporter.client.Exporter");
return composerFactory.createSourceWriter(context, printWriter);
}


Here we use helper classes provided by GWT to create a class with a given packageName and className. We add imports, add interfaces, and then create a SourceWriter to write to the PrintWriter. SourceWriter differs primarily from PrintWriter in that it can keep track of indent level, and indent generated code.

The rest of the exportClass method looks like this:

sw.indent();

// here we define a JSNI Javascript method called export0()
sw.println("public native void export0() /*-{");
sw.indent();

// if not defined, we create a Javascript package hierarchy
// foo.bar.baz to hold the Javascript bridge
declarePackages(requestedType);

// export Javascript constructors
exportConstructor(requestedType);

// export all static fields
exportFields(requestedType);

// export all exportable methods
exportMethods(requestedType);

// add map from TypeName to JS constructor in ExporterBase
registerTypeMap(requestedType);

sw.outdent();
sw.println("}-*/;");

sw.println();

// the Javascript constructors refer to static factory methods
// on the Exporter implementation, referenced via JSNI
// We generate them here
exportStaticFactoryConstructors(requestedType);

// finally, generate the Exporter.export() method
// which invokes recursively, via GWT.create(),
// every other Exportable type we encountered in the exported ArrayList
// ending with a call to export0()

genExportMethod(requestedType, exported);
sw.outdent();

sw.commit(logger);

// return the name of the generated Exporter implementation class
return qualName;
}


This article is too big to delve into every one of this methods, so I will explain just one: exportFields, but first, you will finally have to meet the TypeOracle.

The Oracle that tells you exactly what you need


I created a helper class called ExportableTypeOracle that answers only the questions relevent for my generator:

package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.*;
import com.google.gwt.core.ext.TreeLogger;


public class ExportableTypeOracle {
private TypeOracle typeOracle;
private TreeLogger log;
static final String EXPORTER_CLASS =
"org.timepedia.exporter.client.Exporter";
static final String EXPORTABLE_CLASS =
"org.timepedia.exporter.client.Exportable";
private JClassType exportableType = null;
private JClassType jsoType = null;
private JClassType stringType = null;

public static final String GWT_EXPORT_MD = "gwt.export";
private static final String GWT_NOEXPORT_MD = "gwt.noexport";
public static final String JSO_CLASS =
"com.google.gwt.core.client.JavaScriptObject";
private static final String STRING_CLASS = "java.lang.String";
private static final String GWT_EXPORTCLOSURE = "gwt.exportClosure";

public ExportableTypeOracle(TypeOracle typeOracle, TreeLogger log) {
this.typeOracle = typeOracle;
this.log = log;
exportableType = typeOracle.findType(EXPORTABLE_CLASS);
jsoType = typeOracle.findType(JSO_CLASS);
stringType = typeOracle.findType(STRING_CLASS);
assert exportableType != null;
}

public JExportableClassType findExportableClassType(String requestedClass) {
JClassType requestedType = typeOracle.findType(requestedClass);
if (requestedType != null) {

if (requestedType.isAssignableTo(exportableType)) {
return new JExportableClassType(this, requestedType);
}
}
return null;
}


The first method needed is findExportableClassType. Given a request like "org.foo.bar.MyClass", it looks this type up in the underlying TypeOracle via findType(), and if found, it checks to see if the JClassType that is returned can be assigned to org.timepedia.exporter.client.Exportable. If so, this class is deemed Exportable, and it returns a wrapper around JClassType that can answer more questions about fields, methods, etc.


package org.timepedia.exporter.rebind;

import com.google.gwt.core.ext.typeinfo.JClassType;
import com.google.gwt.core.ext.typeinfo.JField;
import com.google.gwt.core.ext.typeinfo.JMethod;
import com.google.gwt.core.ext.typeinfo.JConstructor;

import java.util.ArrayList;


public class JExportableClassType implements JExportable, JExportableType {
private ExportableTypeOracle exportableTypeOracle;
private JClassType type;
private static final String IMPL_EXTENSION = "ExporterImpl";

public JExportableClassType(ExportableTypeOracle exportableTypeOracle,
JClassType type) {
this.exportableTypeOracle = exportableTypeOracle;
this.type = type;
}



public JExportableField[] getExportableFields() {
ArrayList<JExportableField> exportableFields =
new ArrayList<JExportableField>();

for (JField field : type.getFields()) {
if (ExportableTypeOracle.isExportable(field)) {
exportableFields.add(new JExportableField(this, field));
}

}
return exportableFields.toArray(new JExportableField[0]);
}


For brevity, I only show the getExportableFields() method. This method enumerates over the real ClassType.getFields() method, and for each JField entry, asks the ExportableTypeOracle if it is exportable or not. If so, it creates another wrapper around JField called JExportableField and adds it to the returned results.

But what makes a field exportable?


public static boolean isExportable(JField field) {
return field.isStatic() &&
field.isPublic() &&
field.isFinal() &&
( isExportable(field.getMetaData(GWT_EXPORT_MD)) ||
( isExportable(field.getEnclosingType()) &&
!isNotExportable(
field.getMetaData(GWT_NOEXPORT_MD))));
}


A field is exportable if and only if it is: 1) "public static final", 2) it has a @gwt.export annotation OR its class has a @gwt.export annotation and the field does not have a @gwt.noexport annotation.

Finally, let's look at the exportFields() method.

/**
* For each exportable field Foo, we generate the following Javascript:
* $wnd.package.className.Foo = JSNI Reference to Foo
*
* @param requestedType
* @throws UnableToCompleteException
*/
private void exportFields(JExportableClassType requestedType)
throws UnableToCompleteException {
for (JExportableField field : requestedType.getExportableFields()) {
sw.print("$wnd." + field.getJSQualifiedExportName() + " = ");
sw.println("@" + field.getJSNIReference() + ";");
}
}


Self-explanatory except for the getJSQualifiedExportName() and getJSNIRefernece(). The former method will return the JavaScript package + JavaScript constructor + exported name of the field. So for a field "FOO" on class "org.foo.Bar", this string might be "org.foo.Bar.FOO" if none of the exported names were overriden with an annotation.

The getJSNIReference() function will return a string like "org.foo.Bar::FOO" to reference to the public static final field of that class.

Finally, when done generating, you call SourceWriter.commit(logger), and return the name of the class you generated.

That about wraps up this edition of GWT Demystified. The GWT Exporter source and module is now available on http://code.google.com/p/gwt-exporter

-Ray

7 comments:

Charlie Collins said...

Excellent article, and entire series. Great work.

Unknown said...

Nice post, Ray, if perhaps a bit daunting to chew through. :)

One comment: exporting public static final fields seems somewhat flaky, since you can't prevent JavaScript code from blowing away the copy of the field. (Of course, "DON'T DO THAT" is a reasonable answer -- and what I'd tell someone who used JSNI to overwrite a final field.)

Ray Cromwell said...

Thanks guys.

I ran out of steam during the end, since I had too much other stuff on my plate, so I ended up just using large portions of commented source code for the final article.

In retrospect, a more 'hello world' style generator would have been a better choice, but I couldn't think of anything that was both trivial and useful. :)

-Ray

Ary Manzana said...

Thanks for this great article. The gwt-exporter is exactly what I need in my project.

Ronen said...

Ray,

First off, you are an invaluable source of information for extending and using the gwt compiler. Thanks.

I have a problem that you might help me with:

I'd like to extend the gwt compiler to add to log messages:

1. Class and method names
2. File names and line numbers (of java files)

In addition, I would like to add "entering" and "returning" log messages to all methods.

Is this possible? If yes, how?

As a side note, it would be great to have an AspectJ compatible aop tool added to gwt.

Thanks,
Ronen

Unknown said...

情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣用品,情趣,情趣,情趣,情趣,情趣,情趣,情趣用品,情趣用品,情趣,情趣,A片,A片,A片,A片,A片,A片,情趣用品,A片,情趣用品,A片,情趣用品,a片,情趣用品,視訊聊天室,聊天室,視訊,ut聊天室,聊天室,視訊聊天室,成人電影,

A片,A片,AV女優,色情,成人,做愛,情色,AIO,視訊聊天室,SEX,聊天室,自拍,AV,情色,成人,情色,aio,sex,成人,情色,色情,情色電影,色情網站,av女優,av,自拍,成人,視訊聊天室,視訊交友網,AV女優,成人,聊天室,ut聊天室,av女優

免費A片,美女視訊,情色交友,免費AV,色情網站,辣妹視訊,美女交友,色情影片,成人影片,成人網站,H漫,18成人,成人圖片,成人漫畫,情色網,日本A片,免費A片下載,性愛

色情A片,A片下載,色情遊戲,色情影片,色情聊天室,情色電影,免費視訊,免費視訊聊天,免費視訊聊天室,一葉情貼圖片區,情色視訊,免費成人影片,視訊交友,視訊聊天,言情小說,愛情小說,AV片,A漫,av dvd,情色論壇,視訊美女,AV成人網,情色文學,成人交友,成人電影,成人貼圖,成人小說,成人文章,成人圖片區,成人遊戲,愛情公寓,情色貼圖,成人論壇,色情


免費A片,日本A片,A片下載,線上A片,成人電影,嘟嘟成人網,成人貼圖,成人交友,成人圖片,18成人,成人小說,成人圖片區,微風成人區,成人文章,成人影城,情色貼圖,色情聊天室,情色視訊,情色文學,色情小說,情色小說,臺灣情色網,色情遊戲,嘟嘟情人色網,麗的色遊戲,情色論壇,一葉情貼圖片區,做愛,性愛,美女視訊,辣妹視訊,免費視訊聊天,美女交友,做愛影片

Anonymous said...

(法新社倫敦四日電) 英國情色大亨芮孟的a片下載公司昨天AV片說,芮孟日成人影片前去成人網站世,sex享壽八十二歲;色情這位身av價上億的房地產開發情色電影商,曾經在倫敦推成人網站出第一場脫衣舞表av演。

色情影片
芮孟的財產成人影片估計成人達六億五千萬英鎊(台幣將a片近四百億),由於他名下事業大多分布在倫敦夜生活區蘇活區色情成人因此擁有「蘇活情色視訊之王」日本av的稱號。
部落格

他的成人電影公司「保羅芮孟集團」旗成人網站下發行多a片種情色雜av誌,包括「Razavzav女優leavdvd」、「男性世界」以及「Mayfai情色電影r」。色情a片
a片下載
色情
芮孟情色本名傑福瑞.安東尼.奎恩,父av女優親為搬運承a片包商。芮孟十五歲離開學校,矢言要在表演事部落格業留名,起先表演讀心術,後來成為巡迴歌舞雜耍表演av女優的製作情色人。


許多評論a片成人電影認為,他把情色表演帶進主流社會,一九五九部落格年主持破天荒的脫衣舞表演,後來成人影片更靠著在蘇活區與成人光碟倫敦西區開發房地產賺得大筆財富。


有人形容芮孟是英國的海夫納,地位等同美國的「花花公子」創辦人海夫納。