72
I Use This!
Activity Not Available

News

Analyzed 3 months ago. based on code collected 9 months ago.
Posted about 2 years ago
Posted about 2 years ago
Posted about 2 years ago
I had to move servers a few months back, and in the process something went south with this blog. I've changed a few things, and the blog is alive now again. As a matter of fact, I have little time for NetBSD these days, so don't expect ... [More] many new articles. Just take this as a sign of life. :-) Of course if you think I should add some entry on something here, drop me an email and who knows - maybe I can get things rolling again here? [Less]
Posted over 4 years ago by Kamil Rytarowski
As NetBSD-9 is branched, I have been asked to finish the LLVM sanitizer integration. This work is now accomplished and with MKLLVM=yes build option (by default off), the distribution will be populated with LLVM files for ASan, TSan, MSan, UBSan ... [More] , libFuzzer, SafeStack and XRay. I have also transplanted basesystem GDB patched to my GDB repository and managed to run the GDB regression test-suite. NetBSD distribution changes I have enhanced and imported my local MKSANITIZER code that makes whole distribution sanitization possible. Few real bugs were fixed and a number of patches were newly written to reflect the current NetBSD sources state. I have also merged another chunk of the fruits of the GSoC-2018 project with fuzzing the userland (by plusun@). The following changes were committed to the sources: ab7de18d0283 Cherry-pick upstream compiler-rt patches for LLVM sanitizers 966c62a34e30 Add LLVM sanitizers in the MKLLVM=yes build 8367b667adb9 telnetd: Stop defining the same variables concurrently in bss and data fe72740f64bf fsck: Stop defining the same variable concurrently in bss and data 40e89e890d66 Fix build of t_ubsan/t_ubsanxx under MKSANITIZER b71326fd7b67 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER c581f2e39fa5 Avoid symbol clashes in fs/nfs/nfsservice under MKSANITIZER 030a4686a3c6 Avoid symbol clashes in bin/df under MKSANITIZER fd9679f6e8b1 Avoid symbol clashes in usr.sbin/ypserv/ypserv under MKSANITIZER 5df2d7939ce3 Stop defining _rpcsvcdirty in bss and data 5fafbe8b8f64 Add missing extern declaration of ib_mach_emips in installboot d134584be69a Add SANITIZER_RENAME_CLASSES in bsd.prog.mk 2d00d9b08eae Adapt tests/kernel/t_subr_prf for MKSANITIZER ce54363fe452 Ship with sanitizer/lsan_interface.h for GCC 7 7bd5ee95e9a0 Ship with sanitizer/lsan_interface.h for LLVM 7 d8671fba7a78 Set NODEBUG for LLVM sanitizers 242cd44890a2 Add PAXCTL_FLAG rules for MKSANITIZER 5e80ab99d9ce Avoid symbol clashes in test/rump/modautoload/t_modautoload with sanitizers e7ce7ecd9c2a sysctl: Add indirection of symbols to remove clash with sanitizers 231aea846aba traceroute: Add indirection of symbol to remove clash with sanitizers 8d85053f487c sockstat: Add indirection of symbols to remove clash with sanitizers 81b333ab151a netstat: Add indirection of symbols to remove clash with sanitizers a472baefefe8 Correct the memset(3)'s third argument in i386 biosdisk.c 7e4e92115bc3 Add ATF c and c++ tests for TSan, MSan, libFuzzer 921ddc9bc97c Set NOSANITIZER in i386 ramdisk image 64361771c78d Enhance MKSANITIZER support 3b5608f80a2b Define target_not_supported_body() in TSan, MSan and libFuzzer tests c27f4619d513 Avoids signedness bit shift in db_get_value() 680c5b3cc24f Fix LLVM sanitizer build by GCC (HAVE_LLVM=no) 4ecfbbba2f2a Rework the LLVM compiler_rt build rules 748813da5547 Correct the build rules of LLVM sanitizers 20e223156dee Enhance the support of LLVM sanitizers 0bb38eb2f20d Register syms.extra in LLVM sanitizer .syms files Almost all of the mentioned commits were backported to NetBSD-9 and will land 9.0. As a demo, I have crafted a writing on combining RUMPKERNEL, MKSANITIZER with the honggfuzz fuzzer: Rumpkernel assisted fuzzing of the NetBSD file system kernel code in userland. GDB I've merged NetBSD distribution downstream GDB patches into my local GDB tree and executed the regression tests (check-gdb): [...] Test run by kamil on Mon Sep 2 12:36:03 2019 Native configuration is x86_64-unknown-netbsd9.99 === gdb tests === Schedule of variations: unix [...] === gdb Summary === # of expected passes 54591 # of unexpected failures 3267 # of expected failures 35 # of unknown successes 3 # of known failures 59 # of unresolved testcases 29 # of untested testcases 141 # of unsupported tests 399 Full log is here. This means that there are a lot of more tests and known failures than in 2017-09-05: $ uname -a NetBSD chieftec 8.99.2 NetBSD 8.99.2 (GENERIC) #0: Sat Sep 2 22:55:29 CEST 2017 root@chieftec:/public/netbsd-root/sys/arch/amd64/compile/GENERIC amd64 Test run by kamil on Tue Sep 5 17:06:28 2017 Native configuration is x86_64--netbsd === gdb tests === Schedule of variations: unix [...] === gdb Summary === # of expected passes 16453 # of unexpected failures 483 # of expected failures 9 # of known failures 28 # of unresolved testcases 17 # of untested testcases 41 # of unsupported tests 25 There are actually some regressions and a set of tests that fails probably due to environment differences like lack of gfortran at hand. Full log is here GSoC Mentoring The Google Summer of Code programme reached the end. My mentees wrote successfully their final reports: Enchancing Syzkaller Support for NetBSD, Part 3 Adapting TriforceAFL for NetBSD, Part 3 I'm also mentoring the AFL+KCOV work by Maciej Grochowski. Maciej will visit EuroBSDCon-2019 and speak about his work. Add methods for setting and getting the thread name I've reached out to the people from standards bodies and I'm working on defining the standard approach for setting and getting the thread name. I have received a proper ID of my proposal and I'm now supposted to submit the text in either PDF or HTML format. This change will allow to manage the thread name with an uniform interface on all comforming platforms. Plan for the next milestone Keep enhancing GDB support. Keep detecting ptrace(2) bugs and addressing them. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]
Posted over 4 years ago
Posted over 4 years ago by Kamil Rytarowski
Prepared by Siddharth Muralee(@R3x) as a part of Google Summer of Code’19 As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that ... [More] supports a variety of operating systems including NetBSD. You can take a look through the first report to see the initial changes that we made and you can look at the second report to read about the initial support we added for fuzzing the network stack. This report details the work done during the final coding period where the target was to improve the support for fuzzing the filesystem stack. Filesystem fuzzing is a relatively less explored area. Syzkaller itself only has filesystem fuzzing support for Linux. Analysis of the existing Linux setup Filesystems are more complex fuzzing target than standalone system calls. To fuzz Filesystems we do have a standard operation like mount which comes with system call vector and an additional binary image of the filesystem itself. While normal syscalls generally have a size of a few bytes, sizes of real world Filesystem images is in order of Gigabytes or larger, however for fuzzing minimal size can be used which is in order of KB-MB. Since syzkaller uses a technique called as mutational fuzzing - where it mutates random parts of the input (according to specified guidelines), having a large input size causes delay due to higher I/O time. Syzkaller deals with large images by disassembling them to non-zero chunks of the filesystem image. Syzkaller extracts the non-zero chunks and their offsets and stores it as separate segments and just before execution it writes all the chunks into the corresponding offsets - generating back the new/modified image. Porting it to NetBSD As an initial step towards filesystem fuzzing we decided to port the existing Linux approach of creating random segments to NetBSD. There are a few differences between the mounting process in both the operating systems - the most significant of them being the difference in the arguments to mount(2). Linux: int mount(const char *source, const char *target, const char *filesystemtype, unsigned long mountflags, const void *data); The data argument is interpreted by the different filesystems. Typically it is a string of comma-separated options understood by this filesystem. mount(8) - shows possible arguments for each of the filesystem. possible options for xfs filesystem in linux : wsync, noalign, swalloc, nouuid, mtpt, grpid, nogrpid, bsdgroups, sysvgroups,norecovery, inode64, inode32, ikeep, noikeep, largeio, nolargeio, attr2, noattr2, filestreams, quota, noquota, lazytime, nolazytime, usrquota, grpquota, prjquota, uquota, gquota, pquota, uqnoenforce, gqnoenforce, pqnoenforce, qnoenforce, discard, nodiscard, dax, barrier, nobarrier, logbufs, biosize, sunit, swidth, logbsize, allocsize, logdev, rtdev NetBSD: Int mount(const char *type, const char *dir, int flags, void *data, size_t data_len); The argument data describes the file system object to be mounted, and is data_len bytes long. data is a pointer to a structure that contains the type specific arguments to mount. For FFS (one of the most common filesystems for NetBSD) - the arguments look like : struct ufs_args { char *fspec; /* block special file to mount */ }; Currently, we have a pseudo syscall syz_mount_image which does the job of writing the mutated chunks of the filesystem into a file based on their offsets and later configuring the loop device using vndconfig(8) and mounting the filesystem image using mount(8). Analysis of the current approach One way to create mountable filesystems is to convert an existing filesystem image into a syzkaller pseudo grammar representation and then add it to the corpus so that syzkaller uses it for mutation and we have a proper image. Some of the noted issues with syzkaller approach (as noted in "Fuzzing File Systems via Two-Dimensional Input Space Exploration) : Lack of metadata knowledge - This may lead to corruption of filesystem specific aspects such as checksums. Lack of Context awareness - Syzkaller isn't aware of the status of the filesystem image after a few operations are performed on it. Steps Forward We also spent some time researching possible options to solve the existing issues and developing an approach that would give us better results. Image mutator approach One possible way forward is to actually use a seed image (a working filesystem image) and write a mutator which would be aware of all the metadata in the image. The mutator should be also be able to recreate metadata components such as the checksum so that the image is mountable. An existing implementation of such a mutator is JANUS which is a filesystem mutator written for Linux with inspiration from fsck. Grammar based approach Syzkaller uses a pseudo-formal grammar for representing arguments to syscalls. This grammar can also be modified to actually be able to properly generate filesystem images. Writing grammar to represent a filesystem image is quite a daunting task and we are not yet sure if it is possible but it is the approach that we have planned to take up as of now. Proper documentation detailing the structure of a filesystem image is rather scarce which has led me to actually go through filesystem code to figure out the type, uses and limits of a certain filesystem image. This data then has to be converted to syzkaller representation to be used for fuzzing. One advantage of writing a grammar that would be able to generate mountable images is that we would be able to get more coverage than fuzzing with a seed image, since we are also creating new images instead of just mutating the same image. I am currently working on learning the internals of FFS and trying to write a grammar definition which can properly generate filesystem images. Miscellaneous Work Meanwhile, I have also been working in parallel on improving the existing state of Syzkaller. Add kernel compiled with KUBSAN for fuzzing So far we only used a kernel compiled with KCOV and KASAN for fuzzing with syzkaller. We also decided to add support for syzkaller building a kernel with KUBSAN and KCOV. This would help us have an another dimension in the fuzzing process. This required some changes in the build config. We had to remove the hardcoded kernel config and add support for building a kernel with a config passed to the fuzzer. This move would also help us to easily add support for upcoming sanitizers such as KMSAN. Improve syscall descriptions Improving system call descriptions is a constant ongoing work - I recently added support for fuzzing syscalls such as mount, fork and posix_spawn. We are also planning to add support for fuzzing device drivers soon. Relevant Links Syzkaller Dashboard for NetBSD Syzkaller repository on Github NetBSD docs on setting up syzkaller GSoC'19 proof of work repository Summary We have managed to meet most of the goals that we had planned for the GSoC project. Overall, I have had a wonderful summer with the NetBSD foundation and I look forward to working with them to complete the project. Last but not least, I want to thank my mentors, @kamil and @cryo for their useful suggestions and guidance. I also thank Maciej for his insight and guidance which was very fundamental during the course of the project. I would also like to thank Dmitry Vyukov, Google for helping with any issues faced with regard to Syzkaller. Finally, thanks to Google to give me a good chance to work with NetBSD community. [Less]
Posted over 4 years ago by Kamil Rytarowski
Prepared by Akul Pillai as part of GSoC 2019. This is the third report summarising the work done in the third coding period for the GSoC project of Adapting TriforceAFL for NetBSD kernel syscall fuzzing. Please also go through the first and second ... [More] report. This post also outlines the work done throughout the duration of GSoC, describes the implications of the same and future improvements to come. Current State As of now TriforceAFL has been made available in the form of a pkgsrc package (wip/triforceafl). This package allows you to essentially fuzz anything in QEMU’s full system emulation mode using AFL. TriforceNetBSDSyscallFuzzer is built on top of TriforceAFL, specifically to fuzz the NetBSD kernel syscalls. It has also now been made available as wip/triforcenetbsdsyscallfuzzer. Several minor issues found in the above two packages have now been resolved, and the project restructured. Issues found include: The input generators would also test the the generated inputs, causing several syscalls to be executed which messed up permissions of several other directories and lead to other unwanted consequences. Testing of syscalls has now been disabled as the working ones are specified. Directory specified for the BIOS, VGA BIOS and keymaps for QEMU in the runFuzz script was incorrect. This was because wip/triforceafl did not install the specified directory. The package has now been patched to install the pc-bios directory. The project was structured in a manner where the host was Linux and the target was NetBSD. Since that is no longer the case and the fuzzer is meant to be run on a NetBSD machine, the project was restructured so that everything is now in one directory and there is no longer a need to move files around. The packages should now work as intended by following the instructions outlined in the README. The fuzzer was able to detect a few bugs in the last coding period, details can be found in the last report. During this coding period, the Fuzzer was able to detect 79 unique crashes in a period of 2 weeks running on a single machine. The kernel was built with DEBUG + LOCKDEBUG + DIAGNOSTIC. Work is underway to analyse, report and fix the new bugs and to make the process faster. With an initial analysis on the outputs on the basis of the syscall that lead to the crash, 6 of the above crashes were unique bugs, the rest were duplicates or slight variants, of which 3 have been previously reported. Here are the backtraces of the new bugs found (The reproducers were run with kUBSan Enabled): BUG1: [ 110.4035826] panic: cv_enter,172: uninitialized lock (lock=0xffffe3c1b9 fc0c50, from=ffffffff81a436e9) [ 110.4035826] cpu0: Begin traceback... [ 110.4035826] vpanic() at netbsd:vpanic+0x1fd [ 110.4035826] snprintf() at netbsd:snprintf [ 110.4035826] lockdebug_locked() at netbsd:lockdebug_locked+0x45e [ 110.4035826] cv_timedwait_sig() at netbsd:cv_timedwait_sig+0xe7 [ 110.4035826] lfs_segwait() at netbsd:lfs_segwait+0x6e [ 110.4035826] sys___lfs_segwait50() at netbsd:sys___lfs_segwait50+0xe2 [ 110.4035826] sys___syscall() at netbsd:sys___syscall+0x121 [ 110.4035826] syscall() at netbsd:syscall+0x1a5 [ 110.4035826] --- syscall (number 198) --- [ 110.4035826] 40261a: [ 110.4035826] cpu0: End traceback... [ 110.4035826] fatal breakpoint trap in supervisor mode [ 110.4035826] trap type 1 code 0 rip 0xffffffff8021ddf5 cs 0x8 rflags 0x282 cr2 0x73f454b70000 ilevel 0x8 rsp 0xffff8a0068390d70 [ 110.4035826] curlwp 0xffffe3c1efc556a0 pid 709.1 lowest kstack 0xffff8a006838d 2c0 Stopped in pid 709.1 (driver) at netbsd:breakpoint+0x5: leave db{0}> bt breakpoint() at netbsd:breakpoint+0x5 vpanic() at netbsd:vpanic+0x1fd snprintf() at netbsd:snprintf lockdebug_locked() at netbsd:lockdebug_locked+0x45e cv_timedwait_sig() at netbsd:cv_timedwait_sig+0xe7 lfs_segwait() at netbsd:lfs_segwait+0x6e sys___lfs_segwait50() at netbsd:sys___lfs_segwait50+0xe2 sys___syscall() at netbsd:sys___syscall+0x121 syscall() at netbsd:syscall+0x1a5 --- syscall (number 198) --- 40261a: BUG2: [ 161.4877660] panic: LOCKDEBUG: Mutex error: rw_vector_enter,296: spin lock hel d [ 161.4877660] cpu0: Begin traceback... [ 161.4877660] vpanic() at netbsd:vpanic+0x1fd [ 161.4877660] snprintf() at netbsd:snprintf [ 161.4877660] lockdebug_abort1() at netbsd:lockdebug_abort1+0x115 [ 161.4877660] rw_enter() at netbsd:rw_enter+0x645 [ 161.4877660] uvm_fault_internal() at netbsd:uvm_fault_internal+0x1c5 [ 161.4877660] trap() at netbsd:trap+0xa71 [ 161.4877660] --- trap (number 6) --- [ 161.4877660] config_devalloc() at netbsd:config_devalloc+0x644 [ 161.4877660] config_attach_pseudo() at netbsd:config_attach_pseudo+0x1c [ 161.4877660] vndopen() at netbsd:vndopen+0x1f3 [ 161.4877660] cdev_open() at netbsd:cdev_open+0x12d [ 161.4877660] spec_open() at netbsd:spec_open+0x2d0 [ 161.4877660] VOP_OPEN() at netbsd:VOP_OPEN+0xba [ 161.4877660] vn_open() at netbsd:vn_open+0x434 [ 161.4877660] sys_ktrace() at netbsd:sys_ktrace+0x1ec [ 161.4877660] sys___syscall() at netbsd:sys___syscall+0x121 [ 161.4877660] syscall() at netbsd:syscall+0x1a5 [ 161.4877660] --- syscall (number 198) --- [ 161.4877660] 40261a: [ 161.4877660] cpu0: End traceback... [ 161.4877660] fatal breakpoint trap in supervisor mode [ 161.4877660] trap type 1 code 0 rip 0xffffffff8021ddf5 cs 0x8 rflags 0x286 cr2 0xfffffffffffff800 ilevel 0x8 rsp 0xffff9c80683cd4f0 [ 161.4877660] curlwp 0xfffffcbcda7d36a0 pid 41.1 lowest kstack 0xffff9c80683ca2 c0 db{0}> bt breakpoint() at netbsd:breakpoint+0x5 vpanic() at netbsd:vpanic+0x1fd snprintf() at netbsd:snprintf lockdebug_abort1() at netbsd:lockdebug_abort1+0x115 rw_enter() at netbsd:rw_enter+0x645 uvm_fault_internal() at netbsd:uvm_fault_internal+0x1c5 trap() at netbsd:trap+0xa71 --- trap (number 6) --- config_devalloc() at netbsd:config_devalloc+0x644 config_attach_pseudo() at netbsd:config_attach_pseudo+0x1c vndopen() at netbsd:vndopen+0x1f3 cdev_open() at netbsd:cdev_open+0x12d spec_open() at netbsd:spec_open+0x2d0 VOP_OPEN() at netbsd:VOP_OPEN+0xba vn_open() at netbsd:vn_open+0x434 sys_ktrace() at netbsd:sys_ktrace+0x1ec sys___syscall() at netbsd:sys___syscall+0x121 syscall() at netbsd:syscall+0x1a5 --- syscall (number 198) --- 40261a: BUG3: [ 350.9942146] UBSan: Undefined Behavior in /home/ubuntu/triforce/kernel/ src/sys/kern/kern_ktrace.c:1398:2, member access within misaligned address 0x2b0 000002a for type 'struct ktr_desc' which requires 8 byte alignment [ 351.0025346] uvm_fault(0xffffffff85b73100, 0x2b00000000, 1) -> e [ 351.0025346] fatal page fault in supervisor mode [ 351.0025346] trap type 6 code 0 rip 0xffffffff81b9dbf9 cs 0x8 rflags 0x286 cr2 0x2b00000032 ilevel 0 rsp 0xffff8780684d7fb0 [ 351.0025346] curlwp 0xffffa992128116e0 pid 0.54 lowest kstack 0xffff8780684d42 c0 kernel: page fault trap, code=0 Stopped in pid 0.54 (system) at netbsd:ktrace_thread+0x1fd: cmpq %rbx,8(% r12) db{0}> bt ktrace_thread() at netbsd:ktrace_thread+0x1fd Reproducing Crashes Right now the best way to reproduce a bug detected is to use the Fuzzer’s driver program itself: ./driver -tv < crash_file The crash_file can be found in the outputs directory and is a custom file format made for the driver. Memory allocation and Socket Creation remain to be added to the reproducer generator(genRepro) highlighted in the previous post and will be prioritised in the future. Implications Considering that we have a working fuzzer now, it is a good time to analyse how effective TriforceAFL is compared to other fuzzers. Recently Syzkaller has been really effective in finding bugs in NetBSD. As shown in the below diagrams, both TriforceAFL and Syzkaller create multiple instances of the system to be fuzzed, gather coverage data, mutate input accordingly and continue fuzzing, but there are several differences in the way they work. TriforceAFL Syzkaller Key differences between the two include: KCOV Syzkaller relies on the KCOV module for coverage data in NetBSD whereas TriforceAFL gets coverage information from its modified version of QEMU. VMs Syzkaller creates multiple VM’s and manages them with syz-manager. Whereas TriforceAFL simply forks the VM for each testcase. Communication Syzkaller uses RPC and ssh to communicate with the VMs whereas TriforceAFL uses a custom hypercall. Hardware Acceleration Syzkaller can use hardware acceleration to run the VMs at native speeds, whereas TriforceAFL can only utilize QEMU’s full system emulation mode, as it relies on it for coverage data. These differences lead to very different results. To get a perspective, here are some stats from syzkaller, which can be found on the syzbot dashboard. Bugs Found Upstream Fixed 57 37 20 Comparatively in 1st Weekend of Fuzzing:   Bugs Found Syzkaller 18 TriforceAFL 3 Compared to syzkaller, the number of bugs found by TriforceAFL in the first few days were significantly less, but nevertheless TriforceAFL was able to find variants of bugs found by syzkaller and 1 which was different with simpler reproducers. Going by the stats provided by Syzbot and AFL, Syzkaller does ~80 execs / min whereas TriforceAFL can do ~1500 execs / min on average, Although this statistic is without any of the sanitizers enabled for TriforceAFL and kASan enabled for Syzkaller. TriforceAFL has an advantage when it comes to the fact that it does not rely on KCOV for coverage data. This means it can get coverage data for fuzzing other interfaces easily too. This will be beneficial when we move onto Network and USB Fuzzing. Issues have been found with KCOV where in certain cases the fuzzer lost track of the kernel trace, especially in networking where after enqueuing a networking packet the fuzzer lost track as the packet was then handled by some other thread. Coverage data gathered using TriforceAFL will not be sensitive to this, considering that noise from the kernel is handled in some way to an extent. Efforts were made in the original design of TriforceAFL to make the traces as deterministic as possible at a basic block level. To quote from this article: Sometimes QEMU starts executing a basic block and then gets interrupted. It may then re-execute the block from the start or translate a portion of the block that wasn't yet executed and execute that. This introduced non-determinism. To reduce the non-determinism, cpu-exec.c was altered to disable QEMU's "chaining" feature, and moved AFL's tracing feature to cpu_tb_exec to trace a basic block only after it has been executed to completion. Although nothing has been specifically done to reduce noise from the kernel, such as disabling coverage during interrupts. TriforceAFL runs 1 fuzzing process per instance, whereas Syzkaller can run upto 32 fuzzing processes. But multiple fuzzing instance can be created with TriforceAFL as detailed in this document to take advantage of parallelization. Maxime Villard was also able to rework TriforceNetBSDSyscallFuzzer to now also support compat_netbsd32 kernel. This work will be integrated as soon as possible. On the other hand, Syzkaller has syzbot which can be thought of as a 24/7 fuzzing service. TriforceAFL does not have a such a service or a dedicated server for fuzzing. A service like this will surely be advantageous in the future, but it is still a bit too early to set up one. To truly efficiently utilize such a service, it would be better to improve TriforceAFL in all ways possible first. Targets to be met before a 24/7 fuzzing service is setup, include but are not limited to: Automatic initial Analysis & Report Generation Right now, There is nothing that can automatically perform an initial analysis to detect truly unique crashes and prepare reports with backtraces, this will be required and very helpful. Better Reproducers As mentioned before, the generated reproducers do not take care of Memory Allocation and Socket Creation, etc. These need to be included, if a good C reproducer is expected. Parallel Fuzzing and Management There is no central interface that summarises the statistics from all fuzzing instances, and manages such instances in case of anomalies/errors. Something like this would be needed for a service that will be run for longer periods of time without supervision. Updated AFL and QEMU versions Updated AFL and QEMU might significantly increase executions per second and lead to better mutation of inputs and also decrease the memory requirements. Fuzzing more Interfaces Right now we are only fuzzing syscalls, Network and USB Layers are great future prospects, at least prototype versions can be added in the near future. Future Work Although GSoC will be officially ending, I am looking forward to continuing the development of TriforceAFL, adding features and making it more effective. Some improvements that can be expected include: Fuzzing different interfaces - Network Fuzzing, USB Fuzzing Updating AFL and QEMU versions Better Reproducers Syzbot like service A new repo has been created at https://github.com/NetBSD/triforce4netbsd. Collaborative work in the future will be updated here. Conclusion TriforceAFL has been successfully adapted for NetBSD and all of the original goals of the GSoC proposal have been met, but the work is far from complete. Work done so far shows great potential, and incremental updates will surely make TriforceAFL a great fuzzer. I am looking forward to continuing the work and making sure that this is the case. GSoC has been an amazing learning experience! I would like to thank Maxime Villard for taking an active interest in TriforceAFL and for assisting in testing and development. I would like to thank my mentor Kamil Rytarowski for being the guiding force throughout the project and for assisting me whenever I needed help. And finally I would like to thank Google for giving me this wonderful opportunity. [Less]
Posted over 4 years ago by Christos Zoulas
This article was prepared by Surya P as a part of Google Summer of Code 2019 To begin with where we left off last time, we were able to fix the suse131 package with this commit.This commit adds the GPU-specific bits to the package. And with that ... [More] we had direct rendering enabled and working.I tested it out with glxinfo and glxgears applications. Testing In order to make sure that applications did not break with this commit,I tried Libreoffice and to no surprise everything ran as expected without any hiccups. Then I had to make a choice between porting steam and implementing compat_netbsd32 but since steam had lot of dependencies which needed to be resolved and since implementation of compat_netbsd32 had much more priority I started with the implementation of compat_netbsd32. Implementing compat_netbsd32 DRM ioctls - The Setup For the Setup I downloaded i386 sets from the official NetBSD site and extracted it in the /emul directory. I ran some arbitrary programs like cat and ls from the emulated netbsd32 directory to make sure everything ran perfectly without any problems. I then tried running the 32bit glxinfo and glxgears application and to no surprise it kept segfaulting. I ktraced the application and identified the DRM ioctl that needed to be implemented. Implementing compat_netbsd32 DRM ioctls - The Code There were several functions which were required for the complete working of the compat_netbsd32 DRM ioctl. We implemented each and every function and had the code compiled. We then made sure that the code compiled both as a module and as well as a non module option with which the kernel can be built.I initially tested the code with 32bit glxinfo and glxgears , and the program didn't segfault and ran as expected. Implementing compat_netbsd32 DRM ioctls - Testing In order to test the code I built a test application leveraging the api’s provided in libdrm. It is a very simple application which initializes the DRM connection, setup and draws a gradient on screen and exits. I initially ran it against the native amd64 architecture, but to my surprise the application didn't work as expected. After some hours of debugging I realized that there can be only one DRM master and X was already a master. After exiting the X session and running the application, everything ran perfectly for both amd64 as well as i386 architectures. What is done The Drm Ioctls implementation of Netbsd has been tested and verified The suse131 package has patched and updated (committed) Compat_netbsd32 DRM ioctls has been implemented (Merged) Subsequently DRM ioctls for emulated 32bit linux as well Created a Test GUI Application for the code (yet to PR) TODO Create an ATF for the code and merge it into the tree Read the code, look for bugs and clean it up Port Steam and make it available in NetBSD Conclusion Completing the tasks listed in the TODO is of highest priority and would be carried over even if it exceeds the GSOC time period. Last but not the least I would like to thank my mentors @christos and @maya for helping me out and guiding me throughout the process and Google for providing me with such a wonderful opportunity to work with NetBSD community. [Less]
Posted over 4 years ago by Michał Górny
This report was prepared by Manikishan Ghantasala as a part of Google Summer of Code 2019> This is the third and final report of the project Add KNF (NetBSD style) clang-format configuration that I have been doing as a part of Google Summer of ... [More] Code (GSoC) ‘19 with the NetBSD. You can refer to the first and second reports here: Adding NetBSD KNF to clang-format, Part 1 Adding NetBSD KNF to clang-format, Part 2 About the project ClangFormat is a set of tools to format C/C++/Java/JavaScript/Objective-C/Protobuf code. It is built on top of LibFormat to support workflow in various ways including a standalone tool called clang-format, and editor integrations. It supports a few built-in CodingStyles that include: LLVM, Google, Chromium, Mozilla, Webkit. When the desired code formatting style is different from the available options, the style can be customized using a configuration file. The aim of this project is to add NetBSD KNF support to clang-format and new styles to libFormat that support NetBSD’s style of coding. This would allow us to format NetBSD code by passing `-style=NetBSD` as an argument. How to use clang-format While using clang-format one can choose a style from the predefined styles or create a custom style by configuring specific style options. Use the following command if you are using one of the predefined styles clang-format filename -style= Configuring style with clang-format clang-format supports two ways to provide custom style options: directly specify style name in the -style= command line option or use -style=file and put style configuration in a .clang-format or _clang-format file in the project’s top directory. Check Clang-Format Style Options to know how different Style Options works and how to use them. When specifying configuration in the -style= option, the same configuration is applied for all input files. The format of the configuration is: -style=’{key1: value1, key2: value2, ...}' The .clang-format file uses YAML format. An easy way to get a valid .clang-format file containing all configuration options of a certain predefined style is: clang-format -style=llvm -dump-config > .clang-format After making required changes to the .clang-format file it can be used as a custom Style by: clang-format -style=file Changes made to clang-format The following changes were made to the clang-format as a part of adding NetBSD KNF: New Style options added: BitFieldDeclarationsOnePerLine AlignConsecutiveListElements Modifications made to existing styles: Modified SortIncludes  and IncludeCategories to support NetBSD like includes. Modified SpacesBeforeTrailingComments to support block comments.  The new NetBSD Style Configurations: This is the final configurations for clang-format with modified changes to support NetBSD KNF. AlignTrailingComments: true AlwaysBreakAfterReturnType: All AlignConsecutiveMacros: true AlignConsecutiveLists: true BitFieldDeclarationsOnePerLine: true BreakBeforeBraces: Mozilla ColumnLimit: 80 ContinuationIndentWidth: 4 Cpp11BracedListStyle: false FixNamespaceComments: true IndentCaseLabels: false IndentWidth: 8 IncludeBlocks: Regroup IncludeCategories: - Regex: '^' Priority: 1 SortPriority: 0 - Regex: '^' Priority: 1 SortPriority: 1 - Regex: '^ Status of each Style Option Styles Ready to Merge: 1.Modified SortIncludes and IncludeCategories:      Patch: https://reviews.llvm.org/D64695 Styles needing revision 1.BitFieldDeclarationsOnePerLine:     Patch: https://reviews.llvm.org/D63062     Bugs: 1   2.SpacesBeforeTrailingComments supports Block Comments:     Patch: https://reviews.llvm.org/D65648     Remark:  I have to discuss more on the cases that Block comments are used but where the Spaces are not to be added before them. WIP Style 1.AlignConsecutiveListElements:     Commit: https://github.com/sh4nnu/clang/commit/4b4cd45a5f3d211008763f1c0235a22352faa81e     Bugs: 1 About Styles BitfieldDeclarationsOnePerLine:     Patch: https://reviews.llvm.org/D63062      This style lines up BitField declarations on consecutive lines with correct indentation. Input: unsigned int bas :3, hh : 4, jjj : 8; unsigned int baz:1, fuz:5, zap:2; Output: unsigned int bas : 3, hh : 4, jjj : 8; unsigned int baz:1, fuz:5, zap:2; Bug: Indentation breaks in the presence of block comments in between. Input: unsigned int bas : 3, /* foo */ hh : 4, /* bar */ jjj : 8; Output: unsigned int bas : 3, /* foo */ hh : 4, /* bar */ jjj : 8; Modification for SortIncludes and IncludeCategories:     Patch: https://reviews.llvm.org/D64695     Status: Accepted, and ready to land. Clang-format has a style option named SortIncludes which sorts the includes in alphabetical order.The IncludeCategories Style allows us to define a custom order for sorting the includes. It supports POSIX extended regular expressions to assign Categories for includes. The SortIncludes then sorts the #includes first according to increasing category number and then lexically within each category. When IncludeBlocks  is set to Regroup  merge multiple #includes blocks together and sort as one. Then split into groups based on category priority. The problem arises when you want to define the order within each category, which is not supported. In this modification a new field named SortPriority is added, which is optional. The #includes matching the regexs are sorted according to the values of SortPriority, and Regrouping after sorting is done according to the values of Priority. If SortPriority is not defined it is set to the value of Priority as a default value. Example IncludeCategories: - Regex: ‘<^c/’ Priority: 1 SortPriority: 0 - Regex: ‘^<(a|b)/’ Priority: 1 SortPriority: 1 - Regex: ‘^<(foo)/’ Priority: 2 - Regex: ‘.*’ Priority: 3 Input #include "exe.h" #include #include #include #include #include #include #include Output #include #include #include #include #include #include #include #include "exe.h" As you can observe in the above example the #includes are grouped by different priority and were sorted by different priority. Introduction of this new patch doesn’t affect the old configurations as it will work as the same old SortIncludes if sortPriority is not defined. Refer to Report 2 for detailed examples on this. Modification for SpacesBeforeTrailingComments     Patch: https://reviews.llvm.org/D64695 The SpacesBeforeTrailingComments is modified to support Block Comments which was used to support only line comments. The reason for this is block comments have different usage patterns and different exceptional cases. I have tried to exclude cases where some tests doesn’t support spaces before block comments. I have been discussing in the community for getting to know which cases should be included, and which to exclude. Cases that were excluded due to failing tests: If it is a Preprocessor directive, If it is followed by a LeftParenthesis And if it is after a Template closer AlignConsecutiveListElements     Status: Work In Progress This is a new style that aligns elements of consecutive lists in a nested list. The Style is still in work in progress. There are few cases that needed to be covered and fix few bugs. Input: keys[] = { {"all", f_all, 0 }, { "cbreak", f_cbreak, F_OFFOK }, {"cols", f_columns, F_NEEDARG }, { "columns", f_columns, F_NEEDARG }, }; Output: keys[] = { { "all", f_all, 0 }, { "cbreak", _cbreak, F_OFFOK }, { "cols", f_columns, F_NEEDARG }, { "columns", f_columns, F_NEEDARG }, }; Work to be done: This style option aligns list declarations that are nested inside a list, I would also like to extend this style to align individual single line list declarations that are consecutive. The problem with this style is the case in which there can be different number of elements for each individual. Example: keys[] = { "all", f_all, 0 }; keys2[] = { "cbreak", _cbreak, F_OFFOK }; keys3[] = { "cols", f_columns, F_NEEDARG, 7 }; keys4[] = { "columns", f_columns }; Future Work Some Style Options that were introduced during this GSoC were made in order to meet all the cases in NetBSD KNF. So they may need some revisions with respect to other languages and coding styles that clang-format supports. I will continue working on this project even after the GSoC period on the style options that are yet to be merged and add new style options if necessary and will get the NetBSD Style merged with upstream which is the final deliverable for the project. I would like to take up the responsibility of maintaining the “NetBSD KNF” support to clang-format. Summary Even though officially the GSoC’19 coding period is over, I definitely look forward to keep contributing to this project. This summer has had me digging a lot into the code from CLANG and NetBSD for references for creating or modifying the Style Options. I am pretty much interested to work with NetBSD again, I like being in the community and I would like to improve my skills and learn more about Operating Systems by contributing to this organisation. I would like to thank my mentors Michal and Christos for their constant support and patient guidance. A huge thanks to both the NetBSD and LLVM community who have been supportive and have helped me whenever I have had trouble. Finally a huge thanks to Google for providing me this opportunity. [Less]
Posted over 4 years ago by Leonardo Taccari
This report was written by Naveen Narayanan as part of Google Summer of Code 2019. This report encompasses the progress of the project during the third coding period. You can make sense of the overall progress of the project by ... [More] going through the first evaluation report and second evaluation report. WINE on amd64 Wine-4.4 (released on Mar 2019) is working fine on amd64 and i386. I have been able to use a script as a workaround for the problem of setting LD_LIBRARY_PATH. My patch for setting guard size to 0 and hence, precluding Wine from segfaulting, that got upstreamed, can be found here. I have updated the package to the latest development version of Wine which is Wine-4.13 (released on Aug 2019). I have added support to Wine pkgsrc packages to run tests using make test, and at the time of writing, they are failing. I have also noticed them fail on Linux non-pkgsrc environment and hence, will require further investigation. Initially, they were disabled owing to pkgsrc setting FORTIFY_SOURCE which is a macro that provides support for detecting buffer overflows. In pkgsrc, the wip/wine* packages honor PKGSRC_USE_FORTIFY variable passing _FORTIFY_SOURCE macro accordingly. Programs compiled with FORTIFY_SOURCE substitute wrappers for commonly used libc functions that don't do bounds checking regularly, but could in some cases. Wine unconditionally disables that via their configure script because for some platforms that triggered false positives in the past. However, in my experience, no false positive were found. Running tests on Wine-4.13 throws errors as you can see below: mode64# make test /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M adsldp.dll -p adsldp_test.exe.so sysinfo && touch sysinfo.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 002a:err:winediag:SECUR32_initNTLMSP ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution. 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f150,0x00000001,0x22f120) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EEA0 000000000002B028 000000000022EE9C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022F0C8: stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022FB48: stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so cred && touch cred.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 002e:fixme:cred:CredReadW unhandled type -1 002e:fixme:cred:CredReadW unhandled flags 0xdeadbeef 002e:err:cred:CredWriteW bad username L"winetest" 002e:err:cred:CredWriteW bad username (null) 002e:err:cred:CredWriteW bad username (null) 002e:fixme:cred:CredDeleteW unhandled type -1 002e:fixme:cred:CredDeleteW unhandled flags 0xdeadbeef 002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub 002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub 002e:fixme:cred:CredReadDomainCredentialsW (0x1b3c0, 0x0, 0x22fa50, 0x22f908) stub cred.c:820: Tests skipped: CRED_TYPE_DOMAIN_VISIBLE_PASSWORD credentials are not supported or are disabled. Skipping 002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked 002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt && touch crypt.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_lmhash && touch crypt_lmhash.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md4 && touch crypt_md4.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md5 && touch crypt_md5.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_sha && touch crypt_sha.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so eventlog && touch eventlog.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),(null)) stub 0046:fixme:advapi:OpenEventLogW (L"IDontExist",(null)) stub 0046:fixme:advapi:OpenEventLogW (L"IDontExist",L"deadbeef") stub 0046:fixme:advapi:OpenEventLogW Remote server not supported 0046:fixme:advapi:OpenEventLogW ((null),L"deadbeef") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW (L"",L"Application") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:GetEventLogInformation (0x0, 1, 0x0, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0x0, 0, 0x0, 0, 0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x22fb40) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x22fb40) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 8, 0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x0) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x0) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:BackupEventLogW (0x0,(null)) stub 0046:fixme:advapi:BackupEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,(null)) stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:BackupEventLogW (0x0,L"backup2.evt") stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),(null)) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"idontexist.evt") stub 0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",(null)) stub 0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",L"idontexist.evt") stub 0046:fixme:advapi:OpenBackupEventLogW Remote server not supported 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub eventlog.c:546: Tests skipped: We don't have a backup eventlog to work with 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x0,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000000,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000001,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000002,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000d,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000e,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000007,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa84) stub eventlog.c:479: Tests skipped: No records in the 'Application' log 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup2.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0020,0x0000,0x00000000,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000005,0x00000000,0x1b1e0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ClearEventLogW (0xcafe4242,(null)) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"Wine"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"Wine"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000002,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc1"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc1"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc20") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0001,0x00000004,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc300"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc300"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0002,0x00000007,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc1") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc20"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc20"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0002,0x00000009,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc300") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:err:eventlog:ReportEventW L"First string" 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub eventlog.c:889: Tests skipped: No events were written to the eventlog 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "this name is too long", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x0) stub 0046:fixme:advapi:StartTraceA (0x0, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:ControlTraceA (cafe4242, "wine", 0x1b1e0, 1) stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so lsa && touch lsa.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 004a:fixme:advapi:LsaOpenPolicy ((null),0x22f960,0x000f0fff,0x22f930) stub 004a:fixme:security:GetWindowsAccountDomainSid (000000000022F690 0000000000019078 000000000022F68C): semi-stub 004a:fixme:advapi:LsaEnumerateAccountRights (0xcafe,0x22fa20,0x22f990,0x22f92c) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb10,0x000f0fff,0x22fae8) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb00) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb08) stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so registry && touch registry.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. registry.c:2850: Tests skipped: HKCR key merging not supported registry.c:3146: Tests skipped: HKCR key merging not supported 004e:fixme:winspool:PerfOpen (null): stub 004e:fixme:winspool:PerfCollect L"Global", 0x22ead8, 0x22eabc, 0x22eac0: stub 004e:fixme:winspool:PerfClose stub 004e:fixme:winspool:PerfOpen (null): stub 004e:fixme:winspool:PerfCollect L"invalid counter name", 0x22ead8, 0x22eabc, 0x22eac0: stub 004e:fixme:winspool:PerfClose stub registry.c:4032: Test failed: [ 9] expected 1168, got 2 registry.c:4032: Test failed: [10] expected 1814, got 2 *** Error code 2 Stop. make[1]: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64/dlls/advapi32/tests *** Error code 1 Stop. make: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64 Running programs on Wine You can find obligatory screenshots of Wine-4.4/4.13 on amd64/i386 below: 010 Editor (Professional Text/Hex Editor) on Wine-4.13 (amd64) Notepad++ on Wine-4.4 (i386) Pinball on Wine-4.13 (amd64) How to run Wine on NetBSD/amd64 Compile kernel with USER_LDT enabled. Install kernel. Clone wip repo. cd /usr/pkgsrc/wip/wine64; make install (for 4.4) or cd /usr/pkgsrc/wip/wine64-dev; make install (for 4.13) wine notepad Future Plans Wine requires the kernel option USER_LDT to be able to run 32-bit applications on amd64 - facilitated by WoW64. Presently, this feature isn't enabled by default on NetBSD and hence, the kernel has to be compiled with USER_LDT enabled. I will work on getting the tests to pass and finding a better approach to deal with LD_LIBRARY_PATH issue. In addition to that, I shall work on incorporating a NetBSD VM to Wine Test Bot infrastructure so we can preclude Wine from getting out of shape on NetBSD in the future (work in progress). Summary Presently, Wine-4.4 and Wine-4.13 are working fine on amd64/i386. I have been able to meet all the goals as per my original plan. I would like to attribute the success to the support provided by my mentors @leot, @maya and @christos. I would also like to thank my mentor @maxv for solving the conflict between SVS and USER_LDT kernel options. I intend to maintain the packages wine64/wine64-dev/wine32 for the foreseeable future. As stated above, I shall try to set up a NetBSD VM as Wine Test Bot for purpose of CI. Once again, thanks to Google for enabling me to do what I love. Source Code Commit: Wine Commit: pkgsrc WIP Package: Wine64 Package: Wine64-dev Package: Wine32 [Less]