Thursday, September 1, 2011

Proxying object allocations with Templates and class level new/delete overrides in C++

When working with languages such as C++, that allow access to the OS memory, you almost certainly run into situations where the standard memory model does not suit your needs. I want to describe a technique here that might come in handy in situations where finer grained control over objects allocations is required.

In my particular situation I needed to create my objects in a static memory area, namely a precreated array. The reason for that was a badly designed server application that relied on dynamic object allocations, which lead to memory fragmentation problems. I couldn't introduce too many changes into the existing code, since that would lead to a lot of retesting for which we didn't have the time nor the resources, hence I opted for a less invasive approach. This is the solution I want to describe in this article.

Class level new and delete overrides

As the title of the article states, I used templates and class level new/delete overrides to accomplish this. Lets take a look at how C++ performs object allocation for a moment. At its core, the C++ runtime uses some variation of malloc and free to request memory from the operating system when new and delete are called, respectively. Since we can override those operators on the class level, we can control the allocation process of an object of a particular type. Knowing that, we can implement a solution that will allow us to intercept memory allocations and manage them ourself.

The static member problem

One key aspect of simply overriding new and delete on a class level is that those methods are implicitly static, hence, if we want to reuse (through inheritance) our memory allocation class in another set of unrelated classes we may end up routing calls to the same memory area, which most likely is not what is needed. I call these the static member problem.

Lets take a look at an example.



The slide above shows a possible scenario, where a base class that implements the necessary memory management routines is inherited by two child classes that are not necessarily related. The problem is that since new and delete are implicitly static any member that they access are required to be static as well. In our case, if the memory manager class uses some member as a memory store that member in turn needs to be static as well, and this will lead to any class that inherits from our memory manager class to use the same member as its store as well - not necessarily what we want.

Templates and the static member problem

As described above, simply overriding new and delete on a class level will at best provide an ad-hoc solution, because of the static member limitation described above. There is however a way of creating a generalized version of this solution - we can use templates to overcome this limitation.

A C++ template is a description of a class, a sort of a proto class. Templates specifies the intents of the class and the algorithms that the class implements without imposing the specific types that those algorithms are going to operate on. Templates are full fledged compile and runtime entities - during compilation time the compiler can detect errors related to incorrect type manipulation because its able to resolve the types when the template class is instantiated. The compiler generates concrete instances for every parameter combination and by the time the linker picks up the code, it already is working with concrete classes just as if you'd defined them manually in your code. They are also runtime entities, because by the time the code is executed, the objects created in memory are of the types generated out of the instantiation of your template with a particular template parameter combination.

In other words, simplifying thing a lot, every time you instantiate a template in your code with a different set of template parameters the compiler generates a concrete class based on that instantiation, hence at run time, every instantiation of an object of a particular template instance is an object of a completely different type.

Using templates and new and delete overrides together

We know that static members are class level entities, meaning that a static member is shared across all the objects of a particular type. Since we generate a different class type for every template parameter combination, we are effectively eliminating the static member problem. We can effectively use templates to create a different instance of the memory manager class for every class that requires to use it. Lets take a look at a possible implementation:





Notice the use of a the CAllocationProxy template in the above graphic, which is used to route calls to the memory manager class.

And here is the code for the CAllocationProxy class:

#ifndef ALLOCATION_PROXY_H
#define ALLOCATION_PROXY_H

#include <new>

template < typename TPool, typename TClass >
class CAllocationProxy
{
public:
 CAllocationProxy()
 {
 };

 virtual
 ~CAllocationProxy()
 {
 };

 static void
 InitProxyInstance()
 {
  // Perform required initialization here
 };

 static 
 void* operator new ( size_t nSize )
 {
  void* p = NULL;
  // perform required allocation calls here  
  return p;
 };

 static 
 void operator delete ( void *p, size_t nSize ) throw()
 {
  if ( !p )
  {
   return; // do nothing on null pointer
  }

  // perform dealocation here
 };

private:
 static 
 TPool m_Pool;
};

template< typename TPool, typename TClass >
TPool CAllocationProxy<TPool,TClass>::m_Pool;
#endif // ALLOCATION_PROXY_H

The above class takes two template parameters, TPool and TClass.
  • TPool is our memory allocator - the static member we've been talking about in previous sections. The type passed in itself is not relevant here, however I ended up writing my own memory manager that I would pass as parameter there.
  • TClass is the type of objects that would be allocated by the memory manager. Its not used in the above code, however it could be used to pass the size of the object to be allocated to the memory manager.
The rest is pretty self explanatory.

Lets take a look at how these code can be used:

(pseudocode)
// declare a class A to use the allocation proxy class
class A : public  CAllocationProxy<MemoryManager, A> {
  // implement class...
}

// using class A

// initialize proxy instance...
A::InitProxyInstance();

// all allocations are now done by the proxy... 
A myA = new A(); 
 ....
// deletion is performed by the proxy as well...
delete myA; 

Conclusion

I hope that this technique is of use to anyone in the same situation as I was in - where the amount of changes lead to by other solutions were not acceptable. However, I also believe that this is not the only reason to use this particular approach. I believe that it offers more control over the code than other, more traditional approaches - such as placement new or some sort of static create method, which could potentially allow you to achieve a similar effect. Besides that, this will allow you to reuse the code across many different objects and memory managers.

One word of caution, be ware of class level new and delete macro redefines. Many runtimes and frameworks will override new and delete with macros, to intercept those calls in order to allow more control and enhance error reporting/catching - usually this is only done in debug builds. To fix it use something like the following code:

// undef any new macros
#ifdef new
#define __new new
#undef new
#endif

// undef any delete macros
#ifdef delete
#define __delete delete
#undef delete
#endif 
....
// rest of code here
 ...
// restore new macros
#ifdef __new
#define new __new
#undef __new
#endif

// restore delete macros
#ifdef __delete
#define delete __delete
#undef __delete
#endif

Thanks for reading!

Sunday, August 28, 2011

Apache Wicket and MVP (Supervising Controller)

Apache Wicket is a component based web framework for Java. We've been using it to build the new version of our social platform at my current workplace. Its been an overall pleasant experience up to now, but as always there are several things that do get in your way once in a while, specially when you work with complex rich web interfaces. One of those things is the tight coupling of the hierarchy of your markup to the Java code.

Wicket is somewhat similar to ASP.NET in that it is component based. Wicket as well as ASPX, allows you to work with specific components such as a label, button, textbox, etc... The difference with ASPX is that the markup for those controls is basically free form, you or whoever is building the page needs to provide the markup which can then be bound to particular components in your code.

Say for instance that you have a label, and you want to display some text in that label - lets go with the classic one "Hello World!", the steps required to perform that are pretty straight forward: create the markup file; declare some element that makes sense to display the text in your page, and give it an ID (a wicket:id). Then you create the Label component in you Java code and add() it to the page. Something like this...

MARKUP
<body>  
 ...  
 <p wicket:id="label">[Text goes here]</p>  
 ...  
 </body>  

JAVA

public class TestPage extends Page {
    @Override
    public void onBeforeRender() {
        add(new Label("label", "Hello World!"));
    }
}

Thats it. One of the clear benefits of this is that you can leave the markup to the ones that should be working with it, people who's well versed in designing web pages, and not some daltonic nerd who can't tell the difference of dark pink from clear red or something along those lines.

So moving on, wicket does a few things pretty well and some of them are very handy, like the back button problem, wicket keeps a per page map with all the visited pages stored there, for every single version of the page. Every time you hit the back button wicket restores the correct version of the page, with all the information associated with that version. It also offers an easy way to abstract parts of the page's functionality into panels and into more abstract components that can be reused across you app, among some other amenities like built in ajax support, etc...

Wicket however, does have a few shortcomings. I'm not going to go into all of them here, since none of them are as big and glaring as the one I'm about to describe. In wicket your components have to match your markup hierarchy...

OK, so what does this mean? For the most part, you should be fine. If you're building a few static forms that do not change its layout very often the you probably have nothing to worry about, say for instance some reporting interface for a financial application, or any other non UI centric application. But if your working on some nifty consumer oriented site, then the probability of your UI changing becomes very real and you need to have the freedom to be able to swap it easily - you never know what crazy requirement your design team is going to come up for the next version of the page or the site. In any case, if the UI is more prone to changes than the functionality, you want to be able to change it easily.

Lets go through an example to get a better grasp of the problem. Lets say for instance, that you have a panel that displays the name as a link which when clicked turns into an edit box. All is well until someone decides that its not very intuitive and that now they want an edit link that when clicked shows the edit box, and just to make things a little more exiting, they've changed your layout in such a way that all your previous hierarchy is now gone. Now, if we look at this objectively, your functionality hasn't really changed, you can probably still reuse your Java code if you reassign the wicket ids to the right markup elements (you still click a link and show an edit box), alas wicket won't allow that because your hierarchy needs to change in order to match the new markup. This IMHO, introduces several problems, maintenance is one of them. Also, this scenario is not so bad, but there are others where things can get out of hand so much that you'd probably better using something else - say for instance a site that allows you to customize the layout freely. Arguably you can break your UI into panels just enough to give you a bit more control, but the cost in complexity is not justified in my opinion.

The other problem in wicket is that just as in .NET you mix your UI related logic, with the view state controlling logic and your backend logic, which will make things even more complex to maintain, unless something is done about it. 

Supervising Controller implementation in Wicket:

As the title states this article is about applying MVP pattern to wicket, specifically the Supervising Controller variant of the pattern. According to the definition the Supervising Controller variant differs from the Passive View variant in one key aspect. A Passive View does not share access to the model with the presenter/controller, and all interaction with the view is made through the view interface.
  • In the Passive View the interaction is with a dumb view which is concerned only with the manipulation of the graphical elements and does not update or query the model directly.
  • The Supervising Controller on the other hand shares the model with the view, consequently the view is allowed to query and update the model directly as well as to have some simple state related logic.
Wicket is a model based framework, all wicket components accept a model which they are able to interact with in order to share information among related components and the application. This property in my opinion makes wicket a perfect candidate for the Supervising Controller.

Lets take a look at a possible implementation of the Supervising Controller in Wicket. This pattern requires three components at a minimum to work with, this is the View interface, the Controller, and the Model. Lets see the the implementation:

The presenter:
package com.chronodrones.controllers;

import com.chronodrones.model.IMvpFormModel;
import com.chronodrones.views.IMvpFormView;

import java.io.Serializable;

/**
 * Created by IntelliJ IDEA.
 * User: dryajov
 * Date: 8/28/11
 * Time: 12:24 PM
 */
public class MvpFormController implements Serializable {
    private IMvpFormView view;
    private IMvpFormModel model;

    private boolean editable;

    public MvpFormController(IMvpFormView view, IMvpFormModel model) {
        this.view = view;
        this.model = model;
    }

    public void submit() {
        // perform submit related logic here
        update();
    }

    public void update() {
        if (editable) {
            view.showEditView();
        } else {
            view.showDisplayView();
        }

        editable = !editable;
    }
} 

The view interface:
package com.chronodrones.views;
/**
 * Created by IntelliJ IDEA.
 * User: dryajov
 * Date: 8/28/11
 * Time: 12:23 PM
 */
public interface IMvpFormView {
    void showEditView();
    void showDisplayView();
}

The panel code:
package com.chronodrones.mvp_ajax_form;
import com.chronodrones.model.IMvpFormModel;
import com.chronodrones.controllers.MvpFormController;
import com.chronodrones.views.IMvpFormView;
import org.apache.wicket.ajax.AjaxRequestTarget;
import org.apache.wicket.ajax.markup.html.form.AjaxButton;
import org.apache.wicket.ajax.markup.html.form.AjaxCheckBox;
import org.apache.wicket.markup.html.basic.Label;
import org.apache.wicket.markup.html.form.Form;
import org.apache.wicket.markup.html.form.RadioChoice;
import org.apache.wicket.markup.html.form.TextField;
import org.apache.wicket.markup.html.panel.Panel;
import org.apache.wicket.model.IModel;
import org.apache.wicket.model.Model;
import org.apache.wicket.model.PropertyModel;

import java.util.Arrays;
import java.util.Date;
import java.util.List;

/**
 * Created by IntelliJ IDEA.
 * User: dryajov
 * Date: 8/28/11
 * Time: 12:16 PM
 */
public class MvpAjaxFormPanel extends Panel implements IMvpFormView {
    private Form form;
    private AjaxButton submit;

    private Label name;
    private TextField nameEdit;

    private Label lastName;
    private TextField lastNameEdit;

    private Label dob;
    private TextField dobEdit;

    private Label occupation;
    private TextField occupationEdit;

    private Label gender;
    private RadioChoice genderEdit;

    private List<String> genderList = Arrays.asList(new String[]{"Male", "Female"});

    private MvpFormController controller;

    public MvpAjaxFormPanel(String id, IModel<IMvpFormModel> model) {
        super(id, model);

        controller = new MvpFormController(this, model.getObject());
    }

    @Override
    public void onBeforeRender() {
        super.onBeforeRender();

        IMvpFormModel model = (IMvpFormModel) getDefaultModelObject();

        form = new Form("mvpForm", getDefaultModel());

        name = new Label("name", new PropertyModel<String>(getDefaultModel(), "name"));
        nameEdit = new TextField<String>("nameEdit", new PropertyModel<String>(getDefaultModel(), "name"));

        lastName = new Label("lastName", new PropertyModel<String>(model, "lastName"));
        lastNameEdit = new TextField<String>("lastNameEdit", new PropertyModel<String>(getDefaultModel(), "lastName"));

        dob = new Label("dob", new PropertyModel<String>(getDefaultModel(), "dob"));
        dobEdit = new TextField<Date>("dobEdit", new PropertyModel<Date>(getDefaultModel(), "DOB"));

        occupation = new Label("occupation", new PropertyModel<String>(getDefaultModel(), "occupation"));
        occupationEdit = new TextField<String>("occupationEdit", new PropertyModel<String>(getDefaultModel(), "occupation"));

        gender = new Label("gender", new PropertyModel<String>(getDefaultModel(), "gender"));
        genderEdit = new RadioChoice<String>("genderEdit", new PropertyModel<String>(getDefaultModel(), "gender"), genderList).setSuffix(" ");


        boolean selected = false;
        form.add(new AjaxCheckBox("editForm", new Model<Boolean>(selected)) {
            @Override
            protected void onUpdate(AjaxRequestTarget target) {
                controller.update();
                target.addComponent(form); // repaint form
            }
        });

        submit =new AjaxButton("submit") {
            @Override
            protected void onSubmit(AjaxRequestTarget target, Form<?> form) {
                controller.submit(); // perform submit related logic
                target.addComponent(form); // repaint form
            }
        };

        form.add(submit);

        form.add(name);
        form.add(nameEdit);

        form.add(lastName);
        form.add(lastNameEdit);

        form.add(dob);
        form.add(dobEdit);

        form.add(occupation);
        form.add(occupationEdit);

        form.add(gender);
        form.add(genderEdit);

        add(form);
        controller.update();
    }

    public void showEditView() {
        name.setVisible(false);
        lastName.setVisible(false);
        dob.setVisible(false);
        occupation.setVisible(false);
        gender.setVisible(false);

        nameEdit.setVisible(true);
        lastNameEdit.setVisible(true);
        dobEdit.setVisible(true);
        occupationEdit.setVisible(true);
        genderEdit.setVisible(true);

        submit.setVisible(true);
    }

    public void showDisplayView() {
        name.setVisible(true);
        lastName.setVisible(true);
        dob.setVisible(true);
        occupation.setVisible(true);
        gender.setVisible(true);

        nameEdit.setVisible(false);
        lastNameEdit.setVisible(false);
        dobEdit.setVisible(false);
        occupationEdit.setVisible(false);
        genderEdit.setVisible(false);

        submit.setVisible(false);
    }
}

The Panel Markup:
<?xml version="1.0" encoding="UTF-8"?>
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:wicket="http://wicket.apache.org/dtds.data/wicket-xhtml1.4-strict.dtd">
<body>
<wicket:panel>
    <form wicket:id="mvpForm">
        <span>Edit Information: </span>
        <input type="checkbox" wicket:id="editForm">
        <br>
        <span>Name: </span>
        <span wicket:id="name"></span>
        <input type="text" wicket:id="nameEdit">
        <br>
        <span>Last Name: </span>
        <span wicket:id="lastName"></span>
        <input type="text" wicket:id="lastNameEdit">
        <br>
        <span>DOB: </span>
        <span wicket:id="dob"></span>
        <input type="text" wicket:id="dobEdit">
        <br>
        <span>Occupation: </span>
        <span wicket:id="occupation"></span>
        <input type="text" wicket:id="occupationEdit">
        <br>
        <span>Gender: </span>
        <span wicket:id="gender"></span>
        <span wicket:id="genderEdit"/>
        <br>
        <input type="submit" wicket:id="submit">
    </form>
</wicket:panel>
</body>
</html> 

The model interface:
package com.chronodrones.model;
import java.io.Serializable;
import java.util.Date;

/**
 * Created by IntelliJ IDEA.
 * User: dryajov
 * Date: 8/28/11
 * Time: 12:17 PM
 *
 * Interface defining the model that the Panel will use
 */
public interface IMvpFormModel extends Serializable {
    String getName();
    void setName(String name);
    String getLastName();
    void setLastName(String lastName);
    String getOccupation();
    void setOccupation(String occupation);
    Date getDOB();
    void setDOB(Date dob);
    String getGender();
    void setGender(String gender);
}

The model implementation:
package com.chronodrones.model;

import org.apache.wicket.model.Model;

import java.util.Date;

/**
 * Created by IntelliJ IDEA.
 * User: dryajov
 * Date: 8/28/11
 * Time: 12:18 PM
 */
public class MvpFormModel extends Model<IMvpFormModel> implements IMvpFormModel{
    String name;
    String lastName;
    Date dob;
    String occupation;
    String gender;

    public MvpFormModel(String name, String lastName, Date dob, String occupation, String gender) {
        this.name = name;
        this.lastName = lastName;
        this.dob = dob;
        this.occupation = occupation;
        this.gender = gender;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public String getLastName() {
        return lastName;
    }

    public void setLastName(String lastName) {
        this.lastName = lastName;
    }

    public Date getDOB() {
        return dob;
    }

    public void setDOB(Date dob) {
        this.dob = dob;
    }

    public String getOccupation() {
        return occupation;
    }

    public void setOccupation(String occupation) {
        this.occupation = occupation;
    }

    public String getGender() {
        return gender;
    }

    public void setGender(String gender) {
        this.gender = gender;
    }

    @Override
    public IMvpFormModel getObject() {
        return this;
    }
}

Lets go over the the code for a second to understand what it does. Basically we have a panel that displays information such as the name and last name, date of birth, occupation, etc... When the edit information checkbox is clicked it switches to a form where the information can be edited and submitted. This example uses ajax, however in the attached code there are two implementations of the same form one using ajax and another using regular form submits. Both versions use the same controller to update the state of the view and perform the submit logic.

The controller

The controller part of the above code is in charge of reading the model and updating the view accordingly. The constructor takes in a view parameter of type IMvpFormView and a model of type IMvpFormModel and exposes two public methods - update and submit, which describe very closely the intended use of the controller.

  • The update method calls in to one of the two methods exposed by the view interface showEditView() and showDisplayView(), those methods determine the two possible visual states that the view can be in - edit mode and display mode.
  • The submit method performs submit related logic and updates the view to two of the possible states accordingly. Since this is a very simple demonstration I didn't add much to the submit method implementation, however its evident that you can put there pretty much anything thats not related to the UI itself, like updating your persistence layer, validating the information for semantic consistency, etc..
The View

The view is a wicket panel that implements the IMvpFormView interface, apart from that its a standard as it gets. The interface requires to implement two methods that the controller will consume as described above, those two methods will manage the current view state. The trick here is to realize that the is no controlling logic in the panel itself, its all pretty declarative and manages only its own UI components. However the view also consumes the same model as the controller and provides that model to its child components for querying and updating. The controller is initialized in the constructor of the panel and cached in a class member (controller). I usually use wicket's onBeforeUpdate method to create and add all the child components to the view, since adding them in the constructor produces warnings of various sorts, signaling that not all of the state of the panel/page is initialized. Once all the components are created and added to the panel, the view calls the update method of the controller to trigger the logic required to update the view according to the current state. In our case this will trigger the controller to call into one of the IMvpFormView methods that will either show the display or edit version of the view.

The Model

The model is a simple class that extends wickets Model and implements the IMvpFormModel interface. This interface allows us to share the model among the view and the presenter as well as feeding it to several wicket components.


Conclussion

I realize that there are other ways of building more maintainable and flexible applications in Wicket without the use of the SC pattern, however I also believe that this is one of the cleanest ways to approach several of the the shortcomings you run into with Wicket (and many other frameworks for that matter). Another advantage of this pattern is that it makes your code more testable, arguably wicket is already testable, however this adds another level of testability to you code, you can now separate the businesses logic tests from your wicket/ui tests, I'm not going to go into details here, but suppose you have a form that uploads a file to a server and you need to test that the uploading part of the form logic works without having to initialize all of the wicket related stuff, you can do that now because now you have an additional layer - the presenter, where that logic lives, all you need to do is to mock your view and the model and off you go, no need for a Wicket testing context.

To wrap up, lets list the possible pros and cons of using the Supervising Controller:

Pros:   
  • Clear separation of concerns and intents of the UI, from state and business controlling logic
  • Allows for easy view swapping (useful if you plan on having different versions of the same view; a mobile view and a full view for example)
  • Alleviates wicket's tight coupling of markup hierarchy to code hierarchy by making the view less complex
  • Makes the view more declarative
  • More testable, allows to separate tests into UI related and functionality related
Cons:
  • Imposes a coding style (not bad at all IMHO)
  • May create class bloat
  • May introduce unnecessary complexity into the code

One word of caution, its very easy to abuse this pattern by using it in views that don't require it. This will lead to the two cons mentioned above, class bloat and increased complexity, however, if used correctly it will actually avoid them.

This implementation is very bare bones so don't hold it against me - these was put together in an afternoon, however I do believe that this should be enough to give anyone ideas on how to use the SC pattern with Apache Wicket.

Thats all. Thanks for reading!

The source code is on github.