Mozilla Firefox – The Networking Dashboard. Week 4 and 5

Hello,

Over the past two weeks a lot of things have happened. We had the first evaluation and also a big part of Networking Dashboard it’s almost at an end.

Let’s start with 4th week. Robert, Valentin and I met at the university were we continued working at dashboard and also talked about what we were supposed to do over the next coming weeks. Also we have prepared for the evaluation presentation by putting all the patches over the code, testing all functionalities and establishing what we were going to talk about.

About the presentation we were feeling pleased. Also it was pleasing to see all the other students talking about their projects. I really didn’t know so much until the actually evaluation was held. The aftermath beer and the discussions were good and interesting :)

The next few days that were left of the week Valentin and I decided what I should do next. Of course I had to implement the last diagnostic tool: Proxy Settings Test but what we also wanted to do is to move to the next part of the project – Tracing/Debugging. The main idea here was to implement Honza Bambas’s about:timeline add-on ( http://www.janbambas.cz/firefox-detailed-event-tracer-about-timeline/ ). Unfortunately Honza had other plans with his work, but we weren’t discouraged by this. So, after this, we established that I should get working at proxy settings test diagnostic tool, help Valentin at an older patch that he had worked on – about logging ( Bug 801209 ) and also working some more at Javascript code for UI.

Last week I had only managed to implement Proxy Settings Test Diagnostic Tool – filed as Bug 898237; as DNS Lookup tool, the implementation isn’t ready for a review because I had to wait for Robert’s error bug to be accepted first, in order to use it’s functionality to complete both my diagnostic tool patches. The functionality of this tool was already tested and works at it supposed to do.

The implementation was by far the most interesting because I learnt so much. First of all, the only complain that my mentor had for me at first evaluation was that I wasn’t talking enough with people from Mozilla. So for this patch I decided to first ask them for help, and after talk to Valentin. Also, the implementation itself consists in modifying two .webidl files, one .idl file and some work in C++. This was the easiest part. The hard part was to search for the correct service, functions, headers and idl files.

At the end, as I already said, the tool is working beautifully. The code looks good. I can’t wait to show it at mid term.

Now, I have to start working at logging bug and ask people from Networking module for more functionalities that the Dashboard should have, and also get working at UI.

See you next post!

Fortnightly Post #2.6: Tag me well!

The code review went well, there were some minor fixes that Cătălin Frâncu, my mentor, had to make. But now it’s stable, there are no more errors or warnings and everything seems right. I still have to add some comments to my code as I reckon that good implementation needs good documentation.

I’ve started writing the tag page and there are ideas galore. Some of them still need discussed because in my mind they’re somehow equivocal. Basically, on this page, there will be one of the images that need tagging. There will be some fields which the volunteer has to complete (e.g. the lexeme that is in the tag, the centre of the tag coordinates, the the centre of the pointed area coordinates) and this information will be stored in the database. Of course the most difficult part is to get the coordinates from the image. But fortunately, all of this work is done by plugin which is called jCrop. It lets users create a selection on an image and then it returns the coordinates (x, y) and the size (width, height) of it. With some JavaScript (which I have just started learning), I calculate the coordinates of the selection centre, which populate the input fields.Based on this coordinates we plan to draw the tags and pointing arrows on the image using HTML 5 <canvas> tag. I really enjoy this part of the project as it seamlessly combines server side and client side scripting.

That’s all for now and for a few days henceforth. I will indulge myself with a short vacation.

#3 Mozilla Firefox

Hi,

These weeks I focused on finishing the patches I had sent before, so Ping Diagnostic Tool is now reviewed+ and is waiting for landing. Along with it is a bug that was at the top of our stack because it was affecting the both diagnostic tools which me and Catalin had implemented.

This bug consists in a method that maps a NS_ERROR code to a string, more specific, NS_ERROR_OUT_OF_MEMORY error code is mapped to “NS_ERROR_OUT_OF_MEMORY”. We started with 2 ideas: a dictionary holding the (key, #key) pairs and an array of {key, #key} structures. We kept the second one because the hashtable was filling at runtime and this didn’t give us the expected level of performance. So, I began working on this method and I faced a really big problem, some of the error codes (some pseudo ones) was not unique within the Necko module and our method was returning unexpected results. I filed a bug with this problem and I sent a patch with unique error codes, but we gave up the idea because the module owner decided is not a thing to mess with, it has a lot of problems without localization and, to be honest, some tests crashed.

My work was not useless because out mentor had a great idea of constructing a second array with the duplicated codes and iterating firstly through this one, we get the desired results.

After this fight, I worked on tracing some data. We wanted to know how many bytes Firefox sent and how many it received. At that moment, the Networking Dashboard was keeping evidence of the data transferred through each active socket, but after the socket closed we didn’t know nothing about that amount of data. So, I created two bytes counters as members of the nsSocketTransportService class and, on every socket detach, the counters were updated. In the Dashboard class, I combined this counters with the existing tracing variables and now, about:networking has an improved data tracing feature.

By the time last week finished, I had taken a Telemetry bug and I warmed up for this week. This was very educative, I refreshed my knowledges about creating a new histogram and I found out that Firefox has a feature called DoNotTrack (shame on me, it shipped a long time ago) which lets you express a preference not to be tracked by websites. The guys wanted to know if usage measurements from telemetry line-up with their expectations and if anyone actually opts for “Yes, I want to be tracked”. I’ve sent a patch and now I’m waiting for a feedback because I’m not so sure if I understood right the behaviour they expect.

For each one of these features I’ve implemented some basic UIs for debugging purposes and some demos.

It was a great month, I’ve learned a lot of stuff and tricks (# and ## macro operators w0W!) and I’m eager for next evaluation and beer meeting.

Week III

Hello there,

I hope you’re doing well. These weeks I’ve been busy with working on the user-interface. I finished the logical part with javascript so now I am arranging it (so a lot of CSS). Our interface should contain all the facilities that we’ve implement at the begining. I had to pay very much attention to details and keep in mind that out interface should run on every browser from chrome to IE. And one command were supported by a browser and not by other so you should keep this in mind when you work with web pages.

I have learned about gradient, invisible pixels and other things to make my interface more stylish. I have also worked on some tests for our java application for one of our resources to make it more general.

Anda

 

#5 DexOnline – Romanian Literature Crawler

Hi,

This week I build a stable crawling mechanism, which can crawl only a location of the site, having a better Link-following mechanism which follows only under the start URL. The crawled also has a mechanism which transforms relative links to absolute links.

A big problem which I still have is changing the way I am querying the database from idiorm to the paris library. I could not fix this because paris asks me for classes to build the table from.
e.g. If I have the table ‘CrawledPage’, I would need a class called ‘CrawledPage’ which extends ‘Model’.

Another problem when crawling was that my application parsed anything, even files which are not in html format (like png images). To fix this I added a mechanism which tells me what type of page I’m downloading: text, html, png, etc

I left my crawler running for a while and when I came back I found out that it was the 3rd system resources consummer. After some googling I found out that lost variable references are marked for cleanup and freed when the application is finished or when there’s insufficient system memory. After some more search I found out that newer versions of php let you call the garbage collection explicitly, option which was not present in older php versions..

I have to give credit to the logging mechanism build last week because it helped me a lot so far.

#3 World of USO – Code Refactoring

Howdy,

This week I managed to refactor the views from the interface module. This module contains a lot of form processing views and I thought it will be a burden to refactor them. Thankfully, I found out that Django already had the perfect tools for dealing with forms, the generic class based views FormView, CreateView and UpdateView.

Those classes rely on the Python’s object orientation and multiple inheritance. Therefore, the documentation for them is spread across multiple files and it takes a long time to decipher. You have to go to several other pages to find what attributes and methods each class inherits. Fortunately, I read on Stackoverflow about a very useful site, which does exactly what I needed. It lists all the methods and attributes of each generic class, along with their source code.

The only thing that I’m not sure about this refactor is whether two views (edit_spell and add_spell) handle image upload correctly. I couldn’t make them work because of a glitch with my WoUSO development environment. I think I have an issue with Python Imaging Library.

I had a problem with moving a form from the view to forms.py file. The form was defined inside the view. Consequently, it was generating its fields dynamically when the view got called. When I moved it to another file, it wasn’t generating the correct number of fields anymore. Eventually the solution was overriding the default constructor for that form.

Another interesting thing that I learned during this week was that Python’s super() method is very powerful. It delegates method calls not only to a parent class, but also to a sibling class.

Now it’s time for a short vacation, I will be in Costinești the following week. I’ll keep you posted as soon as I get back to work.

Cheers!

#4 DexOnline – Romanian Literature Crawler

Hi,

This week I build a logging mechanism mainly for exceptions. I had a problem with initialising static variables with non static functions(return value) and the error php gave me was no help at all (it wasn’t expecting ‘(‘ after the function name). Finally, after looking at the variable declaration, it hit me it shouldn’t be declared as static:P.
I also made my crawler nicer:) because one of the sites has figured out that I’m not a browser so I had to find a way to fool it (I changed the user_agent).
Another problem that I encountered was that when I wanted to print to the terminal a new_line, it continued writing on the same line and ‘\n’ was no help at all. After googling for a while I found out that php had a predefined constant named PHP_EOL which did the job.
I also found out how to extract the HTTP code (eg 200, 301, 404, 500). Until now I was using a function made my someone on stackoverflow which was very limited in details (returned true or false). After looking deeply into curl_getinfo($curl_handler) I found out it returns an associative array which at index ['http_code'] contains the http code. This works only for 200 and 300 series. For HTTP code 400 and above, I use curl_errno($curl_handler).

I hope this weeks work solves the part where the crawler doesn’t know what HTTP code the page returns (which made a fool out of me at the RSoC presentation) and I hope I’ll have a better control over my crawler with all the logging going on.
I also hope in a short time to do my first commit on SVN.

Good Luck!

#2 Teamshare – Bug fixing and the event simulator

Hello,

During the third week I have fixed a series of bugs from the user and team configuration file generators and I have started understanding the code behind Teamshare. This process is slow due to the fact that it is mainly written in Java, which is a new language to me, but basic knowledge of XML and Maven is also necessary. In spite of this, I am starting to understand how the code works and I feel I have learned quite a bit.

In the fourth week, besides continuing my code understanding, I started work on a script that simulates filesystem events. These events consist in creating, copying, deleting, moving, removing or renaming files and directories within a given directory. After solving a few problems with the copy event, all the other events were easy to implement and fix.

Using the python library shutil, on the event simulator, in order to perform the copy, move and remove actions, I encountered a rather annoying problem. After implementing the copy functions, I noticed that the program crashed everytime it should have copied a file or a folder. After inspecting the problem, I found out that the copytree function from shutil requires the destination folder must not already exist. After seeking a solution to the problem, I resorted to using os.system to run external commands (cp, mv, rm).

In the following weeks I will perform unit testing on the work I have done and review my event simulator for bugs and improvements.

Fortnightly Post #2: Manage file with style!

Almost done with the elFinder file manager!

I have successfully binded its results of users actions to queries in the database. After completing an action (move, delete, copy, rename) the elFinder (elf, henceforth) adds the name of the command to an array and the results to another array. I created a function that makes queries based on the data stored in those arrays. For example, if a new file is uploaded the script creates a new entry in the table with the path of the added file and the user that completed the action; if a file is moved, it changes its path and so on. This is possible as the elf has a bind option that calls an external function whenever a specific user action is completed.

I have sent the code to be reviewed by my mentors and I’m looking forward to my second commit.

My next task is to create a new page where text in images can be tagged. This is helpful in search engine indexing and will be implemented using jCrop.

FinTP – Configuration Wizzard

Hello,

This is my second post for RSoC. Last time i was telling you how vast this project is and how many things are involved to create a secured and reliable product which can be used by financial institutions.

Until now i have been working on the core part of it and i wrote cpp unit tests for some of the methods that the Transport library contains. This library have all the functions necessary to manage message queueing. As a side note, C++ is incredibly fast but sometimes it’s a bit difficult to read because all that boilerplate code.

As i was telling you in the end of my first post , I was going to work on a GUI for FinTP. A very important part of FinTP are the connectors which put messages in a local queue to be sent to a remote queue manager.
Here is an easy to understand schema i recieved from my mentor.

This application will be dinamically configured through an XML file. The main usage scenario will be something like this:

a. Application -> Load FinTPGui.config: contains user interface components description ( tab, labels, fields… )
b. User -> Choose configuration file type to be generated: read from FinTPGui.config ( tags)
c. User -> Fill in all generated fields
d. Application -> Validate fields (using xmlschema)
e. Application -> Save field values to FinTPGui.config
f. User->Confirm to generate FinTP.config file
g. Application -> Validate section constraints (eg. if last filter for Fetcher has type WMQ)
h. Application -> Save final config file to disk

The first problem I have encountered was how to parse an XML file ? Luckily the qt library already had a class for it called QXmlReader. With it I could parse an XML and generate separate fields in GUI for all xml section/tags.
I use qt 5.x and i have to say i’m impressed of the capabilities of Widgets .

Until next time i will develop some more features for this app, and in the future it will also be available on mobile devices. See you soon.