frysk

I’ve been working on Project Archer for some months now, and it has been pretty interesting. It has also been challenging. There are several deep dark wells of technical knowledge that I’ve had to explore in detail: unwinding, dwarf, debuginfo, and ... [More] exceptions (generation, handling and personality routines). So I’ve been reading about, and stepping through a lot of these areas in GDB this last week. When does a program grow so big that one mortal human cannot work on its entirety? I don’t know the metric, but I bet GDB surpasses it. As I’ve worked on improved C exception handling in GDB, it occurred to me that the different bugs I’ve filed could ultimately be put in one: “Make GDB work better with the GCC unwinder.” As GCC has changed in some areas, GDB has not changed in tandem with GCC. The next or finish commands relying purely on longjmp breakpoints is an example. (If you “next” over a C “throw” statement in GDB you will lose control of the inferior. GDB sets a “longjmp” breakpoint via the “next” command code to re-establish control - but the unwinder for C does not use setjmp/longjmp semantics to switch context. Once resumed, the inferior won’t stop at all, or where expected) So this is a problem. It really irritates me when I lose control of an inferior when debugging. The pain is in proportion to the length of the debugging session. Sometimes I spend hours stepping a process. I’ve cursed a good line on several occasions where this has happened It’s easy to see this negatively, and even easier to write a negative thing about it. But it is a fact of life. So what’s the problem? Well in most areas the longjmp trick will work. It won’t for C exceptions. But this grey area really bothers me. What if there are other areas where expectations do not match? Both GCC and GDB are highly complex programs. They change all the time, and where there is no direct transactional specification (ie debuginfo is written to a specification, so are elf binaries, and so on) the assumptions about how GCC generates code will eventually break. If they break in a big way, they will be fixed - and quickly. But if they break in minor little ways, then the user experience dies as a result of a thousand tiny paper cuts. Or a thousand tiny curses. [Less]

So the Archer task list has been posted and I ended up with c expression parsing. The goal of this task is to enable the user to copy and paste an valid c code during the execution of their program hand have it correctly tokenized, lexed ... [More] , looked-up and ultimately evaluated. My first task was to come up with estimates. Ie how much of this problem can we solve can we arrive at a perfect solution, or do we settle for doing good enough job ? How much work is needed to arrive at that solution ? Can it be done by modifying current gdb code ? Do we need to write something from scratch?.. etc In order to gain more insight into the problem, i decide to pic a couple of pet bugs. In the next post I will talk about my adventures inside gdb. [Less]

As some of you know may know. Red Hat will no longer be working on the Frysk project. The debugger team will instead move to work on gdb. About a month ago we went through a period of examination of the status and goals of Frysk. Long discussions on ... [More] the list and in meetings. During this rexamination the question of  “why not improve on gdb ?” came up, and while a couple of years ago the consensus was that the changes we had in mind were too radical to make to gdb. It is now felt that that this is possible as some of these changes are indeed already underway, and the remaining ones should be incremental changes. I have never worked on gdb so I dont have much opinion on how well things will go in that front but I hope that we can improve the user experience and bring to the community the same cool functionality that we had in mind for Frysk. I would also like to say that it was a pleasure working on Frysk. The code base of Frysk was well designed, and handled extension well. Fixing bugs was always straight forward and did not involve a lot of “fitting square pegs into round holes”. Indeed I think one of our vices was that we spent a lot of time cleaning and refactoring the code base to ensure its extensibility and reusability. Our effort on gdb will be going under the code name Archer. Archer is a branch of gdb with the general goal of creating “the best c debugger in the world”. But more specifically here is a list of our goals: Project Archer Goals So!.. On to new frontiers. Sami [Less]

I have just returned from what was my first GUADEC. It was a great conference, great city, great people, great parties…. basically a great experience all around. I gave two presentations one on Frysk and one on Systemtap. Both went well. The talks ... [More] were pretty cool.. I wanted to go to everyone of them. They were recorded so hopefully we’ll see some links pretty soon. Istanbul is a beautiful city. I hope to go back sometime. We were staying in Sultanahmet a great area to stay in; you will find everything from expensive hotels to cheap, clean student hostels. There is no lack of restaurants, coffee shops, historical sites, or beautiful mosques to visit… oh and rooftop patios with surreal views. GUADEC parties were good, but the boat party was by far the best. Something about being in boat in the middle of the Bosporus brings out the party animal inside of you. The local volunteers at GUADEC and the Turkish people in general are very welcoming. You cannot look at a map twice without someone offering to help. It was also nice to see people from difference cultures having with Overall I thought it was a great conference. Things went smoothly, and it was very productive. Thanks to the organizers who made it seem easy. The image above is a view of the Bosporus from the terrace of the hotel. [Less]

You may have heard that I’ve moved to the Frysk project for a while. In case you’re curious, my incremental compiler project is on hold while I do this. (I’ve lamely not blogged about the gcc summit or other things that have happened lately… I’m ... [More] trying to catch up on this stuff, I hope when I do you won’t mind a bit of old news.) Anyway, we’ve started the process of changing frysk’s direction. You can read about what we’re planning in this post on the frysk list; feel free to respond — we want your input. Essentially, we’re looking at changing some aspects of the implementation, and more explicitly making it be a debugger. [Less]

Load Command The commandline portion of Frysk(fhpd) has two different ways to get a process started down the road to debugging, either via the “fhpd” command itself or by using the “load” command after fhpd is started. Before Frysk can do anything ... [More] with a process, it first must be “loaded”. A user can “load” a process in two different ways. The first way is by loading the process when the Frysk commandline process, fhpd, is activated. For example, if a user wants to debug “ls” and pass it the parameters of “-d /home” it can be done in either of these two ways: [[email protected]] $ fhpd ls [0.0] Loaded executable file: /bin/ls (fhpd) run -d /home running with this command: /bin/ls -d /home Attached to process 27853 Running process 27853 /home (fhpd) or this way: [email protected]] $ fhpd (fhpd) load ls -d /home [0.0] Loaded executable file: /bin/ls (fhpd) run running with this command: /bin/ls -d /home Attached to process 27853 Running process 27853 /home (fhpd) Notice that the user does not have to enter the full path to the executable, Frysk looks for the executable in the paths listed in the user’s $PATH environment variable. The full path to the executable would have to be specified otherwise. The example showed how a single process could be loaded, but the “load” command can handle multiple “loads” if you will. That is, the user is allowed to “load” as many processes as they need to, or they could load the same process multiple times possibly passing a different set of parameters each time. If the user wants to see what processes they have loaded at any point in time they can just issue the “load” command without any parameters and a list will be shown. (fhpd) load ls /usr [0.0] Loaded executable file: /bin/ls (fhpd) load echo abcdefg [1.0] Loaded executable file: /bin/echo (fhpd) load true [2.0] Loaded executable file: /bin/true (fhpd) load Loaded procs path-to-executable [0.0] /bin/ls [1.0] /bin/echo [2.0] /bin/true and, if the user wants to see what arguments were loaded with a particular process: (fhpd) info args The args list for: /bin/ls is….. /usr The args list for: /bin/echo is….. abcdefg The args list for: /bin/true is….. (fhpd) Unload Command Once a process has been loaded, if the user does not want to actually start/run the process or the wrong process was loaded or a typo occurred, the “unload” command allows the user to remove a process from the list. For this purpose the “unload” command has 2 options for removing unwanted processes. The first way is by removing all of the processes by using this command: (fhpd) unload -all This will unload all of the processes that are currently loaded. The second way to remove unwanted loaded processes is to do it individually using the “-t” option like this: (fhpd) unload -t xxx where xxx is the major number to the ptset. For example, suppose you have 3 processes loaded and you want to delete the secomd one in the list. (NOTE The “unload” command works like the “load” command is that if no parameters are entered, a listing of the loaded processes is given.) (fhpd) unload Loaded procs path-to-executable [0.0] /bin/ls [1.0] /bin/echo [2.0] /bin/true (fhpd) unload -t 1 (fhpd) unload Loaded procs path-to-executable [0.0] /bin/ls [2.0] /bin/true Now the previsously-loaded process in slot [1.0], /bin/echo has been deleted from the loaded procs list. Coming soon for “load” Currently the user cannot examine information about a process yet when a process is loaded. Other infrastructure within Frysk is now getting in place where data structures and other environment variables can be interrogated when a “load” command is issued and the debuginfo is loaded for the process. See http://sourceware.org/bugzilla/show_bug.cgi?id=5408. Hopefully this will be implemented within the next few weeks. Next blog subject My next blog subject will be the use of the “start”/”run” command. General Frysk info: http://sourceware.org/frysk Please visit this website for full information about Frysk, how to join its mailing list and where the Frysk developers can be found on IRC. [Less]

I just committed a new utility to frysk, I dont know if anybody would be interested in this but here goes. fdebugdump runs a program and prints out the debug information for the modules that are mapped in at that point. I used to use readelf ... [More] -dbug-dump but that doesnt have a heirachical view, so I wrote fdebugdump. So for example for this program: void print(char *what) { while (*what != '\\0') { write(1, what, 1); what ; } } int main(int argc, char** argv) { // XXX: This forgets to check ARGC. print(argv[1]); print("\\n"); return 0; } $fdebugdump funit-hello module: funit-hello DwTag_COMPILE_UNIT funit-hello.c DwTag_BASE_TYPE unsigned char DwTag_BASE_TYPE short unsigned int DwTag_BASE_TYPE unsigned int DwTag_BASE_TYPE long unsigned int DwTag_BASE_TYPE signed char DwTag_BASE_TYPE short int DwTag_BASE_TYPE int DwTag_BASE_TYPE long int DwTag_BASE_TYPE DwTag_POINTER_TYPE DwTag_BASE_TYPE char DwTag_SUBPROGRAM print DwTag_FORMAL_PARAMETER what DwTag_SUBPROGRAM main DwTag_FORMAL_PARAMETER argc DwTag_FORMAL_PARAMETER argv DwTag_POINTER_TYPE module: /lib64/ld-2.8.so Or if you only want to see functions and parameters for example: $ fdebugdump funit-hello | egrep "SUB|PARAM" DwTag_SUBPROGRAM print DwTag_FORMAL_PARAMETER what DwTag_SUBPROGRAM main DwTag_FORMAL_PARAMETER argc DwTag_FORMAL_PARAMETER argv [Less]

There is some beta/experimental hardware watchpoint code in Frysk 0.3. Give it a try and file bugs. Use the “watch” command from fhpd to access it. Also note these are purely hardware watchpoints, so sizes are 1, 2 and 4 bytes  (and 8 bytes on ... [More] x8664). Teresa is working on some code for 0.4 that allows chaining of watchpoints together to watch bigger spaces. [Less]

For those unfamiliar with what a “frysk” is, it is a new generation open source debugger designed to keep pace with today’s hardware/software advances. For any info about frysk, please visit: http://sourceware.org/frysk. Sorry for the late posting ... [More] of this(since the actual release was a couple of weeks ago), but I feel some very important milestones were met during the month of April that are noteworthy. Each month, usually during the first week, a new release of frysk is published and an announcement is published. This date may vary slightly depending upon what software is being worked on and how close a new enhancement/bug fix is to being checked into the source repo. And so, without further ado: Announcing Frysk 0.3 ———————- http://sourceware.org/frysk/ Frysk is a debugging and monitoring framework being developed using Java and C . It is aimed at providing developers and system administrators with the ability to examine and analyze multi-host, multi-process, and multi-threaded systems while they are running. This is the second release of Frysk. The initial release of Frysk occurred on April 4, 2008 and was tagged as version 0.2.1. Contributors to the 0.3 release were: Andrew Cagney, Thiago Jung Bauermann, Mark Wielaard, Petr Machata, Phil Muldoon, Rick Moseley, Sami Wagiaalla, Stan Cox and Teresa Thomas. Here are some of the improvements that were incorporated during April: - Exported a prototype of low level watchpoint api on IA32,X8664 (PPC* will be covered by IBM) - Implemented the watch command which exposes the watchpoint api in fhpd. - Updated various frysk man pages. - Created ProcRunUtil and re-based fcatch,fstep,ferror ontop of it. - Added ability to kill procs from fhpd. - Solidified passing parameters to run command and reusing history. - Added support for elf symbol look-ups. - Removed the CDT parser from frysk. - Rewritten ftrace to use more frysk infrastructure. - Adapted frysk symbol search code so that it can be used by breakpoints, as well as expression evaluation. - Fully implemented sysroot functionality in frysk. - Improved and tested stepping, particularly stepping through signal handlers. - Imported a newer version of upstream elfutils. - Fixed breakpoints to work correctly through forks. Known limitations: some test that are working in-tree fail when installed. To download this release, go to: ftp://sourceware.org/pub/frysk/frysk-0.3.tar.bz2 or check for an update in your local GNU/Linux distro. Rick 2008-05-23 [Less]

I am happy to announce that frysk version 0.2.1 has been released. Downloaded it here, or just ‘yum install frysk’ In this release… lots of good stuff: command line utilities: fcatch - catch and print the stack of a crashing process fcore - extract ... [More] a core file from a running process fdebuginfo - list debug-info requirements of a process fdebugrpm - install debug-info requirements of a process ferror - catch and back-trace error calls fstack - print each thread’s stack ftrace - trace a processes system and library calls and a few more. Prototype command line debugger: fhpd Prototype visual debugging and monitoring tool (frysk) Known limitations: work-flow limited to live processes (examining core files, or creating processes is possible but very non-intuitive). About the version number: We have not been making regular releases of frysk, mainly because a lot of the work was going into the infrastructure of frysk so there weren’t many user visible features. Now that we will be making regular releases we wanted to bump the version number to communicate the huge progress that we have made from 0.0.1 to current version. There were several suggestions: Go to 0.9 then start doing 0.9.1, 0.9.2, etc for minor releases or, go to 0.8 then 0.8.1, 0.8.2, etc, but finally we decided to do 0.2, 0.3, 0.4….0.11, 0.13 etc Enjoy [Less]