In our case we were interested in the hql, sql and time taken for them. We were also running into some issues with the session size, so had a piece to capture that too. The whole approach is based on these simple steps:

1. Putting in a flow identifier
2. Inject our own SessionStatsCollector
3. Wire it via JMX
4. Get callbacks for the original sql

Flow identifier
The utility of this approach depends on how easily can we pin point each flow. The url along with the thread id is a good starting point. However if the urls are kind of the same and the request parameters have a special token, we can use that as our flow discriminators. This token is injected into the NDC for the later part of the tool to pick it up and arrange diagnostic information.

Inject SessionStatsCollector
Hibernate does not allow any hook points to the session stats collector, so to get our custom stats collector in, we used a simple cglib proxy for it. The proxy intercepts only queryExecuted and closeSession methods. While queryExecuted is used to capture the hql, time and number of rows, closeSession callbacks helps us log all the collected information together.

Wire it via JMX
Since the original mbean does not support logging flows, the new jmx bean exposes one method profileFlows which takes the flow pattern and a boolean to indicate if the session size should be logged. Determining the session size is a heavy operation, so on most cases we may not want to do so. There is another method to reset the bean which would revert to the original statistics collector from our souped up version.

Sql callbacks
We were interacting with our dba group, for whom the hql was like greek, so the original sql was important to us, since the original stats implementer does not get any callbacks for this, we injected our own entity interceptor which gives a callback to the stats collector for the sql.

So in short with about 8 classes, we can create something which gives us some indicators of the flow performances. The performance and memory overhead introduced by this is nominal and can be turned off at any point.

The classes for this are easy to code, if laziness gets in your way, post a comment and will upload it.

The general convention of writing your model classes is as following:

public class NameObj extends BaseModelData {

     public String getFirstName() {
          return this.get(“firstname”);
     }

     public void setFirstName (String name) {
          this.set(“firstname”, name);
     }

}

public class Person extends BaseModelData {

     public NameObj getName() {
          return this.get(“name”);
     }

     public void setName(NameObj name) {
          this.set(“name”, name);
     }
     //so on
}

RPC Interface method

     public Person getPersonList(args);

The above code points to one key convention especially when using GXT. We declare getter and setter for concrete types we want to deal with, but we need not want to declare data members for them as everything is being sent in the BaseModelData map.

Now the pitfall!! Since the NameObj was not declared as a data member of the Person class and only Person class is defined to be part of the RPC interface, GWT compiler would not add the NameObj class to RPC whitelist i.e. while serializing RPC response, GWT will most probably complain with the following error.

Caused by: com.google.gwt.user.client.rpc.SerializationException: Type
'com.myclassNameObj was not included in the set of types which can be
serialized by this SerializationPolicy or its Class object could not
be loaded.

A similar issue is reported on the following link as well.
http://osdir.com/ml/GoogleWebToolkit/2009-04/msg01287.html

Thus the GWT compiler smartly works on to find which are the essential classes which need to be serialized as it has the direct implication on amount of javascript compiler would generate and would be required to be downloaded on the browser. Since we were using GXT, we needed to set data in BaseModelData as GXT classes would work when data comes in the base model data map.

The current workaround which we came up with is a way to fake with GWT by declaring unused variables for the classes as following:

public class Person extends BaseModelData {

     @SuppressWarnings(“unused”)
     private NameObj _nameObj

     public NameObj getName() {
          return this.get(“name”);
     }

     public void setName(NameObj name) {
          this.set(“name”, name);
     }
     //so on
}

This may not be the best way to deal with this issue, but as of now, I could not find any other approach. Anyone is welcome to suggest a better alternative.

I would publish another article soon on using dynamic Intenationalization with GWT by using a smarter way than writing a huge array. But that discussion merits a separate article.

“You must override hashCode in every class that overrides equals. Failure to do so
will result in a violation of the general contract for Object.hashCode, which will
prevent your class from functioning properly in conjunction with all hash-based
collections, including HashMap, HashSet, and Hashtable.”

Another Important Line from the Effective Java “In hashCode method you must exclude any fields that are not used in equals comparisons,
or you risk violating the second provision of the hashCode contract”(To refer to contract look into Object.java (javadoc))

I want to add to this and say :

Always compareTo method should reflect same logic as you have in equals.

Why so ?

If you will override hashCode then true your hash-based collections will work but what about collections which are not hash-based (you will ask are there any.).Yes collections like TreeSet(This class implements the Set interface, backed by a TreeMap instance),TreeMap(Red-Black tree based implementation of the SortedMap interface) are not hash-based.

How do they Work then ?

These collections depend on compareTo method which is available in Comparable Interface or compare Method which is available in Comparator Interface. So either these collections can be given Comparator instance or the objects being added in these collections implement Comparable Interface.

Why do the need compareTo or compare ?

Remember First Lecture on Binary Trees.There is a root node and it has child which can be either left of it or right of it. Now how is it decided in TreeMap, well you might have guessed it by now.These compareTo and compare methods are used for it.In Order to decide whether new element has to be added (seacrhed for) in left of or right of the root node depends on the outcome of  compareTo, if outcome is negative integer then in left otherwise in right.(Or if they are equal then change the value of that node, as set doesnt allow duplicates)

What will happen if i screw up this method ?

Lots of bad things will happen:

You will never find your Object which you put in the Collection ?

Well its dangerous can you explain it in more detail. Here is the case you have a equals method which is not in sync with compareTo method then contains() method invoked with the equal object will return false. So one contract is the if two objects are equal then compareTo must return “Zero”.

From the Javadoc of compareTo

“It is generally the case, but not strictly required that (compare(x, y)==0) == (x.equals(y)). Generally speaking, any comparator that violates this condition should clearly indicate this fact. The recommended language is “Note: this comparator imposes orderings that are inconsistent with equals.”

I dont know why they didnt make it required. It should have been a required condition as otherwise contains() operation will be screwed on collection. You are not always going to do contains check with the same object which is already in Collection.You might have the Equal object (The Object  for which equals() return true.) and you want to check if this exist in collection and want to find the object which exists.Typical use case can be in Hibernate you have a transient object and you want to find persistent instance from the collection and then sync the properties from transient to persistent object.

How should we implement compareTo then ?

Well you need to satisfy this condition (compare(x, y)==0) == (x.equals(y)) at any cost. One way is you can use CompareToBuilder and append all the fields which are in equals.

Out of Curiosity if equals condition you mentioned is satisfied but for all others results are biased i mean i am always growing tree in one direction say always compareTo returns greater Then Zero for every object. Will it screw up Performance ?

Nice , so you have finally data structures running in mind. Well it has been affected had it been just a Binary Tree.But its smarter than that, its a Red-Black Tree.

“The constraints of Red Black tree enforce a critical property of red-black trees: that the longest path from the root to any leaf is no more than twice as long as the shortest path from the root to any other leaf in that tree. The result is that the tree is roughly balanced. Since operations such as inserting, deleting, and finding values require worst-case time proportional to the height of the tree, this theoretical upper bound on the height allows red-black trees to be efficient in the worst-case, unlike ordinary binary search trees.”

As per javadoc of TreeMap

“This implementation provides guaranteed log(n) time cost for the containsKey, get, put and remove operations. Algorithms are adaptations of those in Cormen, Leiserson, and Rivest’s Introduction to Algorithms.”

Still didnt understand ? But How remeber you said that it will keep adding in one direction ?

Well enough of spoon feeding. Open TreeMap. java and look into put method. You will find some method like fixAfterInsertion().It will keep balancing the tree after every insert.So it will be invoked after every Insert.

Dude You are driving me crazy.Do you think I am dumb.Isn’t fixAfterInsertion method take time ?

Well that’s beauty of Red-Black Tree.fixAfterInsertion takes O(log(n)), but as it happens after insertion so total time will be O(log(n)) +O(log(n)) which is O(2log(n)) which is anyway O(log(n)).The rotateRight and rotateLeft are constant time operations.You understand constant means O(1).

Never mind coming back to original topic ,another important para which mandates requirment of compareTo and equals to be consistent.

“Note that the ordering maintained by a sorted map (whether or not an explicit comparator is provided) must be consistent with equals if this sorted map is to correctly implement the Map interface. (See Comparable or Comparator for a precise definition of consistent with equals.) This is so because the Map interface is defined in terms of the equals operation, but a map performs all key comparisons using its compareTo (or compare) method, so two keys that are deemed equal by this method are, from the standpoint of the sorted map, equal. The behavior of a sorted map is well-defined even if its ordering is inconsistent with equals; it just fails to obey the general contract of the Map interface.”

I want to see from my own Eyes ?

Download the Test Project from here.

Thanks. Hope you enjoyed reading and learnt some thing.

The reason I have mentioned GXT also in the title is because of the similarity in issue we would face as both use GWT compiler in the background. Further I came across these problems when we started integrating GXT with our existing web application.

Handling authentication

A typical requirement of any web application is to check every request is coming from an authentic source. This involves checking either a session attribute kind of thing which is set once the user has logged in or using request headers or data to validate the user credentials. Since any decent sized application would have multiple RPC interfaces exposed (especially if different pages of an application are GWT enabled instead of the entire application), we need a kind of base servlet which is extended by all RPC servlet implementations in the application.

Once you architect your servlets into this design, the next important aspect is to hook in the authentication logic so that GWT sends the callback to appropriate method in the base servlet before executing any method in the interface implementations. Fortunately GWT already gives you such methods. The method onBeforeRequestDeserialized in RemoteServiceServlet can be overridden in the base servlet, to provide the basic authentication code of the application.

Similarly to add any custom post processing after a RPC method is executed, one can override, onAfterResponseSerialized method.

Consider the scenario when authentication fails or the user session has timed out. Typically one likes to throw some exception like AuthenticatonException. But the RPC interface method ideally should not declare such exceptions in the throws clause of the methods as it’s not really part of the method contract.

This is the first pitfall we would land up in the GWT world! Since the authentication exception is not declared to be thrown from the interface, GWT RPC framework would not add it to its Serialization Policy.

The workaround which we figured out was to inject our own serialization policy for the given exception.

Following are the implementation steps

1. Create a Map with Class object as key and Boolean as value (indicating following class is to be serialized)
2. Add the Authetication Exception to the Map. Remember, all its parent classes have to be added explicitly to the serialization policy map. E.g. If  AuthenticationException extends RuntimeException then add RuntimeException, Exception and Throwable classes to the serialization policy map.
3. Declare the StandardSerializationPolicy object with the above map as the constructor argument.

   //code snippet

   Map<Class<?>, Boolean> exceptionMap = new HashMap<Class<?>, Boolean();
   
   exceptionMap.put(AuthenticationException.class, true);
   
   exceptionMap.put(RuntimeException.class, false);
   
   exceptionMap.put(Exception.class, false);
   
   exceptionMap.put(Throwable.class, false);

4. Override doUnexpectedFailure method of RemoteServiceServlet and if exception received in AuthenticationException, prepare the response string using encodeResponseForFailure method from RPC until class passing the custom serialization policy. (Internally GWT code uses the same code with standard generated serialization policy).

   String responseStr = RPC.encodeResponseForFailure(null, e, preprocessingExceptionPolicy);
   
   RPCServletUtils.writeResponse(getServletContext(), getThreadLocalResponse(), responseStr, false);

That’s it. This would allow you to throw exception which are not declared by RPC interface from the callback methods.

- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {

SetupWizard *wizard = [[SetupWizard alloc]init];

[self.navigationController pushViewController:wizard animated:YES];

-(void)loadPreferences

{

NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];

[textField setText: [userDefaults stringForKey:@"url"]];

if([userDefaults integerForKey:@"whichNumber"] == 0)

cell1.accessoryType = UITableViewCellAccessoryCheckmark;

else

cell2.accessoryType = UITableViewCellAccessoryCheckmark;

[sw setOn:[userDefaults boolForKey:@"switch1"]];

[slider setValue:[userDefaults doubleForKey:@"slider_preference"]];

}

- (void)viewDidLoad {

[self loadPreferences];

}

You should read part 1 of this post if you are not familiar with application preferences. You can find it here .
 

A child pane is nothing but another plist file. So to start with, open the same project “MyTestProject”, right click on Settings.bundle and select “add new”. From the “Other” tab select plist file as shown below:

Name the file as “Child.plist”. You will notice that child.plist file is not added in the Settings.bundle. You need to add it inside Settings.bundle which will be done from the terminal.

To do so, open Terminal and navigate to your projects directory. Execute “ls” on shell to cross verify that you are in the correct directory. Following should appear on the Terminal:

Now copy child.plist file into the Settings.bundle by following command: “cp child.plist Settings.bundle“. Cross verify by navigating into Settings.bundle directory and executing “ls”. child.plist file should show up.

Close the terminal now and come to xCode again. You will now find child.plist inside Settings.bundle. Double click Child.plist and add items to make it look like this:

We have created the new child plist file, now we need to add a reference to it in the root.plist file. Open root.plist and add 2 new items to make it look like this.

Build and run the application , go to the settings and select your project MyTestProject. It shold look similar to the following:

In next post I am going to access these preferences from the MyTestProject and also allow the user to change them from the View itself.

When we bring in Cardinality in Association we get One-to-One, One-to-Many and Many-to-Many.In later two One-to-Many and Many-to-Many we use Collection to represent many Side.For example: An Order have many OrderDetail’s.

We have these associations represented in Relational DBMS using foreign keys.So from any O-R framework we expect that to take care of these associations in smartest way possible.As these can easily become a performance bottle neck for the application.Some time you may end up loading lot of Data into memory, and sad part is you are never going to use them. 

So here is what i think we expect from Collections in O-R:

1. Ability to map the Collections to DB

2. Ability to Load them

3. Ability to traverse smartly

4. Ability to add elements to  collection

5. Ability to remove elements from collection

Some points from above I will try to explain in detail from Hibernate Perspective.I will be skipping mapping part that is well explained in Hibernate Reference Documentation.

Let’s start with which collections we generally want to map :

1. Set

2. List

3. Map

there could be special cases where we want to preserve Order (which can be insertion or sort)

1. SortedSet

2. LinkedList

3. SortedMap

One other kind of collection can be an array.But generally we dont use them in our domain models.Well there are reasons from O-R point also against them like lazy-loading,optimized dirty checking cannot be achieved as we cannot wrap them or byte code instrument them.

Collections and Hibernate

Persistent collections are treated as value objects by Hibernate. ie. they have no independent existence beyond the object holding a reference to them. Unlike instances of entity classes, they are automatically deleted when unreferenced and automatically become persistent when held by a persistent object. Collections can be passed between different objects (change “roles”) and this might cause their elements to move from one database table to another.

Hibernate “wraps” a java collection in an instance of PersistentCollection. This mechanism is designed to support tracking of changes to the collection’s persistent state and lazy instantiation of collection elements. The downside is that only certain abstract collection types are supported and any extra semantics are lost. Now before jumping in PersistentCollection code lets try to think what do we need this wrapper to do:

Lazy Loading

          It need to load actual data only “when” needed. So what is this “when”

                  a.) Doing  size() operation and isEmpty Check

                            Whenever this operation will be invoked PersistentCollection will check if collection has been loaded or not.If not then it will have another check whether this collection has been mapped as extra-lazy or not. If its extra-lazy then it will only do a “select count”  sql to get the size rather then loading the collection. If its not marked extra-lazy then it will load the collection from database and execute operation on actual collection.

                 b.) Doing a contains check

                          For this operation also we need to check whether Collection has been loaded or not. If its not loaded and marked extra-lazy then it will do a “select 1 ” to find out whether this element exists or not.So this will save us loading of the collection and a simple sql will suffice.If its not marked extra-lazy then it will load the collection and execute operation on actual collection.

                  c.) Getting iterator                  

                         For this operation no matter whether its marked extra-lazy or not it will need to initialize the collection from DB. But here it cannot return iterator of actual Collection.It will create a proxy Iterator.As it will need to keep track of remove operation on iterator.

                  d.) Adding some element in collection

                       This operation will have different implementations based on whether we are having a List or Set. As Set cannot allow duplicates. So question finally boils down on can we do an add without loading the collection. Well yes for List we can do that.But for Set we need to do a contains check internally, whether that element already exists or not which is similar to contains check i described above. 

                       If collection is loaded then we can directly do add on the collection and mark the collection dirty. Otherwise it queues the operation to be synced with DB at later stage. Again some kind of lazy behavior. Hibernate calls these operations DelayedOperations and execute them immediatedly after loading the collection whenever its is loaded.

                  e.) Removing some element from collection

                      This opertaion implementation is going to be similar to a Set add. As we need to check whether this element actually exists in DB or not. So if its marked extra-lazy we need to do existence check in DB without loaidng the collection. If its not marked then we need to load the collection and then do the operation on actual collection. Similar to add this can also have lazy behavior and it can be delayed till we load the collection by queuing this operation. Also it needs to mark the collection dirty after this opertaion.

                 f.)  clear operation on collection

                        This opertaion implementation is very simple. If Collection has not been initialized yet.(It needs additional check whether its an Inverse Collection and whether its Orphan Delete or not). It will queue the operation. 

                         Otherwise it will execute the operation. Here hibernate does on smart check of isEmpty as if collection isEmpty it doesn need to clear. As without doing this check if it does clear then it would have needed to mark the collection dirty which would have been wrong. As clearing an Empty collection is not making Collection dirty.

                   What is “Dirty” ?

                   I referred to marking Collection dirty a lot in all operations. What affects does this have ? dirty Field will be used by Flush operations on Session, flush of the Entity Holding the Collection. And this flag is the one which is going to trigger sync of Collection state with the DB.

So we discussed two things one is Lazy Loading of Collection and other is Delayed Operations.

There are lot of details which i am going to discuss for Collection treatment in Hibernate. Till now i discussed Lazy Loading. This is going to be a series of blog Entries on Hibernate Collections. Next Articles in the series ares going to be:

1.  Fetching Collections from DB

2. Sync of Collection State with DB

3. Collections and Caching

4. Treatment of Cardinality in Collection

5. Different kind of Collections