Weekly update #13 [2/9-8/9]

This is technically my final update. I have finished with the testing and docs. I have also done some refactoring efforts of the phpma code as well as the server code. I still have some things to do but not much. I want the phpma branch to get merged soon.

I refactored the access control in the server. I also refactored the phpma code into a library and fixed failing tests. I added the documentation to all models and components. I left the controllers out for now since most of the functions are very simple. I may add some after the soft deadline. 

As for the error reporting server a few things are left like the server is not currently strong enough against a hostile attacker. I sanitize everything. and cakephp has a good security model but I believe I can make it stronger against hostile input.

I created 2 pull requests but they still haven’t been merged. I will try to have them merged this week. Next week I will be in Germany so I am not sure how much work will I be able to do there at first.

It has been a great summer working on this project. I am happy its almost complete 🙂

Weekly update #12 [26/8-1/9]

This week I have finished writing tests for the error reporting server. I have done tests for controllers and models. I am not going to test views or helpers because it would make the tests very fragile if we change a few html tags or classes. I have 100% coverage on almost all the covered files.

As for the phpmyadmin component. I refactored the functions I wrote into a library that I include. I also renamed functions to follow conventions and wrote the doc blocks for those functions.

For next week I should be documenting the error reporting server codebase. I will probably also add the copyright notice to the required files. There are some refactoring I wanted to do in the access control module. Also I am planning to do something useful to the state like add other states and provide a way to change state.

Weekly update #10 [12/8-18/8]

We are nearing the end of the project. This week was graphs. I created pie charts that showed summaries of the incidents in a report. I also added two more summaries so that almost anything that could be summarizable is summarized.

I had used previously a graphing library but it turned out not to be free for commercial use. Since I am not sure what type of use is this I looked for an opensource library that was more flexible and found one that was released under the MIT license.

I also added syntax highlighting in the stack traces to make it easier to see the code snippets. Furthermore I fixed a bug where if the error occurs in error_report.js file then I wouldn’t accept it. I also added another characteristic for grouping error reports which is pma_version. two incidents belong to the same report only if they have the same pma_version.

I created a pull request for the changes here. I still have not prepared the new seed file. This has a new look for the website and I need a new update from the community on the look of the project. I will be crafting an email once I create the seed file.

As for next week I have planned for adding graphs for the entire website like reports per day as well as frequency of reports per pma_version, as well as other summaries that I may find interesting.

Weekly update #8 [29/7-4/8]

This week is the first week of a two week task to handle related reports and grouping of similar reports. It is also the week that we had our first deployment.

I deployed in order to get more feedback on how everything is going. Sadly the responses were not exactly overwhelming but I have a somewhat good view about how I have decided to proceed.

This week I worked on the js line translator and added url sanitization to most urls in the error report. This helps with two things. First is the increased privacy of the reporters, and second is that there is less difference between similar bugs just because they happened at different hosts.

I have also changed the current schema so that we have reports and incidents. I have moved some of the functionality to the incidents model. I have also changed the views and helpers to handle the new schema. However I faced some problems in how to display certain parts of the incidents. since the incidents do not have a view I need to display the relevant information in the reports view. The most problematic pieces of info is the steps leading to the error and the stacktrace.

They need to be displayed but caution must be taken since they may be 1000s of them. I am currently displaying all the steps at the bottom of the report. As for the stack traces I have not decided how I am going to handle them. I have an idea but I am putting it off incase another idea comes to me. The idea is to have php compare the stack trace to all the stacktraces of the related incidents in the database to find if it is the same as another stack trace of a previous incident, otherwise it is marked in the database as a new stacktrace and is displayed in the reports view.

Another problem is displaying all the different stacktraces. I am not exactly a great designer so I might not be able to create a view that is usable but doesn’t waste a lot of space in the page. We will see about it but I am not sure yet.

I have also yet to handle the manual marking of reports as the same. So for next week I am going to handle that, as well as add a view for the stack traces and maybe even create an html view for incidents and hopefully end up with something better than the last time I tried it.

I will not yet merge my current work into the master branch since it is not finished yet. I will wait till the task is finished to submit a pull request so that the deployment server gets the entire story at once.

Speaking of deployment, I was also able to get the checkstyle errors from around 900 to 43 or 57 depending on who you ask. This was quite impressive. I will hopefully continue on this trend.

Also I might take a couple of days off for Eid at the end of next week. I will try to finish next week’s work before that. I am not exactly behind on my schedule so I don’t think that would be a problem

Weekly update #3 [24/6-30/6]

This week I have finished the client side component and once the api between the client and the server is complete I will request merging into master. This should be the last week of me working on the client side component.

My system now currently catches any errors that occur and offer to submit the error report to our servers. It uses file_get_contents or curl depending on availability and respects the proxy settings set by the user.

I wrap all global functions starting with PMA_ I also wrap the callback for AJAX.registerOnload as well as the handler for $.fn.on since it is used excessively throughout the code.

I have tested the code in the latest versions of browsers available on linux (so no IE or safari yet) and it works ok.

I have started on the server side component and created a repo. I have setup the server on my computer. Once I make things more portable I will probably push the initial version of the server early this week. A lot of things are currently dependant on my current setup so it is of no use to push them to a public repo for now.

I will be using cakephp as my development framework. I would have rather used Ruby on Rails but I figure it would be easier on the developers who already know the language to use a php framework.

This week my task is to basically create the server. I have gotten a head start already. I will be using the extra time to take on some of the tasks of the next week so that I would be ahead of schedule.

Weekly update for week #2 [17/6-23/6]

This is the end of my first week of GSoC. I am very happy that I have finished the tasks outlined in this week’s schedule fully. This is the first week of my two weeks in the implementation of the client side component of my error reporting module. So by next week we should be finished with the client side completely and start on the server side.

This week my tasks as per my schedule were:

“create the function that takes an exception and gets all the required info, then anonymizes it. A view is created with the relevant data asking the user to authorize the sending of the report to our servers.”

I have added a handler for the “window.onerror” which collects the necessary info and according to the user settings either sends the error report automatically or ignores it or asks the user what to do. If the user choses to see the error report a modal dialog is opened showing the content of the report as well as asking the user for the steps that lead to the error. Once the user accepts sending the error report the javascript instructs the server to send the error report to our servers.

There is a request being made to the error reporting server but that is not yet implemented so as of yet it should just fail silently. I made sure that all the strings in both the php part and the javascript part are translatable. I have made the javascript respond according to the user preferences like automatic sending or ignoring silently. I have collected a lot of relevant info about the error, however I have not anonymized anything. I did not see a part of the error report that may be considered private however my mentor may suggest a candidate for anonymization once he tries the feature.

For next week my task includes adding of try and catch statements to parts of code so as to get a stacktrace in some cases since this is not possible in “window.onerror”. Also I will try to tackle another problem which is that the line number that is retrieved either in the stacktrace or in “window.error” is for the entire concatenated file of javascripts. This is not very useful as it is not easily reproducible by the developers and is very vague in the sense it is not easy to figure out where this line number is.

When I was writing this report an idea came to me to solve the line number in the concatenated file problem. I can write a bash script that given the get_script.js.php request params and the line number it can output the file which has the relevant number and output the correct relative line number. This is still not very elegant and I will ask the phpmyadmin developers opinions in the mailing list soon.

As for adding try and catch to the functions I have decided that for starters I can wrap them around all functions starting with PMA_. My mentor also suggested I add “AJAX.registerOnload” callback to the list since it is important. I may also wrap other functions manually if I deem them important enough that a large part of the pages uses them or if they do a complicated enough task. but for now it is just the methods I have mentioned. The wrapping is done at run-time so that I will not modify the code of these functions so that this feature can be turned off easily.

I shall do a feature detection to see if a stacktrace is supported by the browser first before trying to wrap these functions in try and catch blocks since it may be useless if the browser doesnt support it. Another problem might be is that if I wait till the document is ready before I do the code wrapping any exception may not be caught by the try and catch blocks however if start too early then the javascript files may not be parsed and some of the functions may have not been defined yet. I still have to research how to do this and what is the best approach.

This is enough for the week. Today I am taking the rest of the day off for a personal matter and I shall start on next week’s tasks tomorrow

First Week with phpmyadmin

Today is the official starting date for my gsoc project with phpmyadmin. My tasks this week include catching exceptions and collecting necessary info as well as showing a modal dialog with the required info and asking for the user’s permission to send the report while observing the stored configuration settings related to it.

I still donot know who to collect that information in the minified js code. I have asked my mentor for help as well as petitioned for unminified js in production. I googled and couldn’t find a good solution for the problem other than source maps which sadly are not yet implemented in most browser versions.

I will start with the other tasks now and even take on some tasks from next week and leave this part till I know how to solve it. Hopefully I will stumble upon the answer soon.