#3 Apache HBase – Extending the current web UI

I believe that the things on the past two weeks went pretty well.

 

I managed to solve most of the bugs on the shell. Currently there is only one bug which occurs when the user tries to evaluate scripts that contain infinite loops.
To resolve this problem I will try to create a daemon which loads the ruby sources and I will try to count the evaluation. In this way I will be able to kill the process if the evaluation
takes too long.

 

Besides solving the bugs I’ve done these shell improvements :
  • build “resize” method (this one allows the user to resize the shell)
  • build “timeout” method (this one allows the user to extend the request timeout)
  • build “start” and “stop” methods (these will be used if the shell access will be based on a whitelist)
  • there are three options to display the output :
      1. paginated output (behaves like the unix command “more“)
      2. scrollable output (without history)
      3. scrollable output (saving shell history)
  • the user can enable or disable the webshell configuring hbase-site.xml file.

 

Also I exposed on the interface information about the cluster state which will be used to debug problems that will occur. For this I created a JS which use a AJAX request to get information from a servlet. The servlet instantiates a java class which evaluates “hbck” script, captures its output and returns it to the script. When the response is complete I expose the information on the front page. In this way the call method is asynchronous and the information is exposed only when it’s complete.

 

So I think that I sticked to the schedule. I still have to solve that bug, but I’ve already created that daemon and the engine that runs it and captures its output. However there are some details that I hope I will solve by Tuesday.

#2 Apache HBase – Extending the current web UI

For me, the last two weeks have passed too quickly and I had to get used to a lot of things and also to make some major decisions. The initial plan was to find a terminal emulator compatible with Apache license and connect it to the REST server, but at the end of the first week I had to change the project’s perspective. I decided that by using the REST server I will not be able to create a shell which can do the same things as the IRB, so I had to find a way which implies using the IRB’s sources.

After a thorough research of terminal emulators I choose to use termlib library [1] . I found many solid terminal emulators (like shellinabox, jcubic, gateone and others) but I believe that it would be easier to use this library. It has implemented many useful methods and its documentation is very well structured. Even with this it was hard to realize a connection between the REST server and the user because the server did not respond to any HTML requests. First thing I thought was that maybe I made syntax mistakes, but after checking the server logs I figured that there was nothing wrong with my HTML requests. These arrived safely to the server, were interpreted correctly , but the server did not send back any response. I had problems with this for a while until I had a talk with my mentor and I decided that even if I find the problem, the REST server will never be able to give me the same functionality as the IRB.

So I had to change my plan and try to use the IRB’s sources. The new idea was to realize an shell endpoint in the Master UI by which components communicate. In the shell page I would create a terminal instance, I would capture the user input and send it to the endpoint, the endpoint would evaluate the command by running its source and return the result to the shell’s page. The endpoint was a class which extends HttpServlet and which communicates with the shell page by doGet and doPost methods. This instantiate a JRuby engine , loads the IRB sources and evaluates the command using engine.eval(“command”, context). 

The first problem that I encountered was that the HTML response sent to the shell page was always empty. It took me a little until I realized that the servlet runs on a different port than where the ajax request is coming from which violates the Same Origin Policy for ajax requests and thus the browser won’t process the ajax response. The solution was to return JSONP or to set the HTTP Access-Control headers. I chose the last one and I added the following line which allows everyone to use the servlet response.setHeader(“Access-Control-Allow-Origin”, “*”) .

The next problem I had slowed my work even more : I realized that I couldn’t get the output of all commands using the engine.eval method so I had to wrote a Ruby class which evaluates the command and captures the output from the stdout. Since I never used Ruby before it took me too much time to construct that class. There were many different new concepts in Ruby which I had to take into account but eventually I managed to figure out how to realize that class.

Now I have to resolve a few bugs and to move on to the next stage of the project :

  •   expose in Java information about cluster status

[1] - http://www.masswerk.at/termlib/

#1 Apache HBase – Extending the current web UI

Hello there! My name is Claudiu and this summer I will try to improve the HBase interface. For this purpose I will :

  • include the shell in the interface;
  • attempt to fix some issues which affect the loading time;
  • add more metrics and administration operations to the interface;
  • expose more information in the Master status;
  • implement login and authorization features.

Unfortunately the first week was not very productive for me because I had difficulties compiling the sources. I also had to get familiar with technologies that I was not familiar with (e.g. Jamon, JSP, Ajax and Jetty).

Now I am trying to find a terminal emulator (preferably written in JS) that is compatible with Apache and I will try to connect it to Stargate (the REST server bundled with HBase [1] ).

I believe things went a little slow because it is the first time I participate in an open source project. I hope that next time I will have more things to tell you. See ya!

[1] http://wiki.apache.org/hadoop/Hbase/Stargate