NetBSD

I've been actively working on reducing the delta with the local copy of sanitizers with upstream LLVM sources. Their diff has been reduced to less than 2000 Lines Of Code. I've pushed to review almost all of the local code and I'm working on ... [More] addressing comments from upstream developers. LLVM changes The majority of work was related to interceptors. There was a need to cleanup the local code and develop dedicated tests for new interceptors whenever applicable (i.e. always unless this is a syscall modifying the kernel state such as inserting kernel modules). Detailed list of commits merged with the upstream LLVM compiler-rt repository: Split getpwent and fgetgrent functions in interceptors Try to unbreak the build of sanitizers on !NetBSD Disable recursive interception for tzset in MSan Follow Windows' approach for NetBSD in AlarmCallback() Disable XRay test fork_basic_logging for NetBSD Prioritize the constructor call of __local_xray_dyninit() (investigated with help of Michal Gorny) Adapt UBSan integer truncation tests to NetBSD Split remquol() from INIT_REMQUO Split lgammal() from INIT_LGAMMAL Correct atexit(3) support in MSan/NetBSD (with help of Michal Gorny investigating the failure on Linux) Add new interceptor for getmntinfo(3) from NetBSD Add new interceptor for mi_vector_hash(3) Cast _Unwind_GetIP() and _Unwind_GetRegionStart() to uintptr_t Cast the 2nd argument of _Unwind_SetIP() to _Unwind_Ptr (reverted as it broke "MacPro Late 2013") Add interceptor for the setvbuf(3) from NetBSD Add a new interceptor for getvfsstat(2) from NetBSD A single patch landed in the LLVM source tree: Swap order of discovering of -ltinfo and -lterminfo (originated by Ryo Onodera in pkgsrc) Patches submitted upstream and still in review: Add interceptors for the sha1(3) from NetBSD Add interceptors for the md4(3) from NetBSD Add interceptors for the rmd160(3) from NetBSD Add interceptors for md5(3) from NetBSD Add a new interceptor for nl_langinfo(3) from NetBSD Add a new interceptor for fparseln(3) from NetBSD Add a new interceptor for modctl(2) from NetBSD Add a new interceptors for statvfs1(2) and fstatvfs1(2) from NetBSD Add a new interceptors for cdbr(3) and cdbw(3) API from NetBSD Add interceptors for the sysctl(3) API family from NetBSD Add interceptors for the fts(3) API family from NetBSD Implement getpeername(2) interceptor Add new interceptors for vis(3) API in NetBSD Add new interceptor for regex(3) in NetBSD Add new interceptor for strtonum(3) Add interceptors for the strtoi(3)/strtou(3) from NetBSD Add interceptors for the sha2(3) from NetBSD Patches still kept locally: ASan thread's termination destructor MSan thread's termination destructor Interceptors for getchar(3) API (might be abandoned as FILE/DIR sanitization isn't done) Incomplete interceptor for mount(2) (might be abandoned as unfinished) This month I've received also a piece of help from Michal Gorny who improved the NetBSD support in LLVM projects with the following changes: [unittest] Skip W+X MappedMemoryTests when MPROTECT is enabled [cmake] Fix detecting terminfo library Changes to the NetBSD distribution I've reduced the number of changes to the src/ distribution to corrections related to interceptors. Document SHA1FileChunk(3) in sha1(3) Fix link sha1.3 <- SHA1File.3 Define MD4_DIGEST_STRING_LENGTH in Correct the documentation of cdbr_open_mem(3) Plan for the next milestone I will keep upstreaming local LLVM patches (less than 2000LOC to go!). This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

I have presented the state of NetBSD sanitizers during two conferences in the San Francisco Bay Area: Google Summer of Code Mentor Summit (Mountain View) and MeetBSDCa (Santa Clara, Intel Campus SC12). I've also made progress in upstreaming of our ... [More] local patches to LLVM sanitizers and introducing generic NetBSD enhancements there. The Bay Area I took part (together with William Coldwell - cryo@) in the GSOC Mentor Summit as a NetBSD delegate. I've presented during the event a presentation with a quick introduction to NetBSD, track history of GSoC involvement and the LLVM Sanitizers work with a stress of sanitizers. NetBSD and userland & kernel SANITIZERS The MeetBSDCa conference is a continuation of the MeetBSD conferences from Poland. I took part there as a speaker talking about Userland Sanitizers in NetBSD. I've also presented the state of virtualization in NetBSD during a discussion panel. Additionally I've prepared a lightning talk about NetBSD Kernel sanitizers and quick status update from The NetBSD Foundation. Unfortunately the schedule was last minute changed (introduction of BSD history talk in the slot of lightning presentations) and the closing ceremony had different proceeding. Nonetheless, I'm sharing these additional quick presentations. Bug detecting software in the NetBSD userland: MKSANITIZER NetBSD kernel sanitizers MeetBSDCa 2018: The NetBSD Foundation update During the former conference it was a great opportunity to meet people from other Open Source projects, a lot of them are in interaction with NetBSD developers during the process of upstreaming local support patches. During the latter conference it was an opportunity to meet BSD people and people closer to hardware companies. Upstreaming process of LLVM Sanitizers I've upstreamed a number of patches to the LLVM source tree. The changes can be summarized as: Further reworking the code and approaching the state of installing of sysctl*() inteceptors. Fixing or marking failing or hanging tests in the sanitizer test-suites. Adapting definitions of syscalls and ioctl(2) operations for NetBSD 8.99.25. Detailed list of commits merged with the upstream LLVM compiler-rt repository: Update ioctl(2) operations for NetBSD 8.99.25 Update generate_netbsd_ioctls.awk for NetBSD 8.99.25 Diable test suppressions-library for NetBSD/i386 Disable BufferOverflowAfterManyFrees for NetBSD Mark breaking asan tests on NetBSD Switch getline_nohang from XFAIL to UNSUPPORTED for NetBSD Mark vptr-non-unique-typeinfo as a broken test for NetBSD/i386 Mark breaking sanitizer_common tests on NetBSD Handle NetBSD alias for pthread_sigmask Cast the return value of _Unwind_GetIP() to uptr Mark interception_failure_test with XFAIL for NetBSD Disable ASan test asan_and_llvm_coverage_test for NetBSD Adapt ASan test heavy_uar_test for NetBSD Mark breaking TSan tests on NetBSD with XFAIL Cleanup includes in sanitizer_platform_limits_netbsd.cc Regenerate syscall hooks for NetBSD 8.99.25 Update generate_netbsd_syscalls.awk for NetBSD 8.99.25 Handle pthread_sigmask in DemangleFunctionName() Drop now hidden ioctl(2) operations for NetBSD Handle NetBSD symbol mangling for tzset Handle NetBSD symbol mangling for nanosleep and vfork Mark test/tsan/getline_nohang as XFAIL for NetBSD Disable the GNU strerror_r TSan test for NetBSD Mark test/tsan/ignore_lib5 as unsupported for NetBSD Mark intercept-rethrow-exception.cc as XFAIL on NetBSD Disable failing tests lib/asan/tests on NetBSD Skip unsupported MSan tests on NetBSD Mark 4 MSan tests as XFAIL for NetBSD Mark MSan fork test as UNSUPPORTED on NetBSD Reflect the current reality and disable lsan tests on NetBSD Use PTHREAD_STACK_MIN conditionally in a test Remove remnant code of using indirect syscall on NetBSD Don't harcode -ldl test/sanitizer_common/TestCases Disable TestCases/pthread_mutexattr_get on NetBSD Fix Posix/devname_r for NetBSD Unwind local macro DEFINE_INTERNAL() Introduce internal_sysctlbyname in place of sysctlbyname Frequently asked question People keep asking me about rationale of some design decisions in sanitizers and whether something could be done better. One of such places is to reuse more of libc internals and to not keep bypassing it whenever possible. The motivation to keep redoing the same work for NetBSD is to keep close to the upstream (mostly Linux & Android) source code with a minimal delta between NetBSD vs others support. Doing some operations in a more convenient way is tempting, but it's a danger that someone will need to keep maintaining a larger diff, especially since upstream developers will focus on their own OSes rather than trying to adapt their patches for potentially alternative approaches. Plan for the next milestone I will keep upstreaming local LLVM patches (almost 2500LOC to go!). This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

I presented the state of NetBSD sanitizers during EuroBSDCon 2018 held in Bucharest, Romania. I gave two talks, one covered userland sanitizers and the other one kernel sanitizers. Unfortunately video recordings from the conference are not ... [More] available, but I've uploaded my slides online: LLVM Sanitizers in the NetBSD userland Taking NetBSD kernel bug roast to the next level: Kernel Sanitizers Besides participating in the conference and preparing for the travel and talks I've been researching the libunwind port to NetBSD and further integration of Lua. The libunwind port from the nongnu project has been approached to passing 22 out of 33 tests and the current blocker is the lack of signal trampoline handling or annotation. A signal trampoline is a special libc function, registered into the kernel, that is used as a helper routine to install and use signal handlers. Backtracing the function call stack is not trivial. We need to either annotate the assembly code in the trampoline with DWARF notes or handle it differently inside an unwinder. I wrote a toy application using the newly created Lua binding for the curses(3) library. The process of writing the Lua bindings resulted in detecting various bugs in the native curses library. A majority of these bugs have been already fixed with aid of Roy Marples and Rin Okuyama, though they are still waiting for merge. I intend to keep working on the bindings in my spare time, but a shortcoming is that there are a lot of API functions (over 300!), and covering them all is time consuming process. Meanwhile, I've made progress in the upstreaming of local LLVM patches. I've finally upstreamed to switch of indirect syscall (syscall(2)/__syscall(2)) to direct libc calls. Plan for the next milestone I will visit the GSoC Mentor Summit & MeetBSDCa in October (California, the U.S.). In the time besides the conference I will keep upstreaming local LLVM patches (almost 3000LOC to go!). This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

This was my first big BSD conference. We also planned - planned might be a big word - thought about doing a devsummit on Friday. Since the people who were in charge of that had a change of plans, I was sure it'd go horribly wrong. The day before ... [More] the devsummit and still in the wrong country, I mentioned the hours and venue on the wiki, and booked a reservation for a restaurant. It turns out that everything was totally fine, and since the devsummit was at the conference venue (that was having tutorials that day), they even had signs pointing at the room we were given. Thanks EuroBSDCon conference organizers! At the devsummit, we spent some time hacking. A few people came with "travel laptops" without access to anything, like Riastradh, so I gave him access to my own laptop. This didn't hold very long and I kinda forgot about it, but for a few moments he had access to a NetBSD source tree and an 8 thread, 16GB RAM machine with which to build things. We had a short introduction and I suggested we take some pictures, so here's the ones we got. A few people were concerned about privacy, so they're not pictured. We had small team to hold the camera :-) At the actual conference days, I stayed at the speaker hotel with the other speakers. I've attempted to make conversation with some visibly FreeBSD/OpenBSD people, but didn't have plans to talk about anything, so there was a lot of just following people silently. Perhaps for the next conference I'll prepare a list of questions to random BSD people and then very obviously grab a piece of paper and ask, "what was...", read a bit from it, and say, "your latest kernel panic?", I'm sure it'll be a great conversation starter. At the conference itself, was pretty cool to have folks like Kirk McKusick give first person accounts of some past events (Kirk gave a talk about governance at FreeBSD), or the second keynote by Ron Broersma. My own talk was hastily prepared, it was difficult to bring the topic together into a coherent talk. Nevertheless, I managed to talk about stuff for a while 40 minutes, though usually I skip over so many details that I have trouble putting together a sufficiently long talk. I mentioned some of my coolest bugs to solve (I should probably make a separate article about some!). A few people asked for the slides after the talk, so I guess it wasn't totally incoherent. It was really fun to meet some of my favourite NetBSD people. I got to show off my now fairly well working laptop (it took a lot of work by all of us!). After the conference I came back with a conference cold, and it took a few days to recover from it. Hopefully I didn't infect too many people on the way back. [Less]

Peter Wemm's writeup about using acme.sh for FreeBSD.org served as inspiration, but I chose to do a few things different: using DNS alias mode with sub-domains dedicated to ACME verification delegating the sub-domains to the servers where the ... [More] certificate will be needed using bind on the servers where the certificate will be needed (where it was running as resolver already anyway) using dns_nsupdate (i.e. dynamic DNS) to add the challenge to the ACME subzone. Appropriately restricted, that gives the following addition to named.conf on the target server (with an update key named acme-ddns): options { .... allow-update { localhost; }; .... }; zone "acme-www.pkgsrc.org" { type master; file "acme/acme-www.pkgsrc.org"; update-policy { grant acme-ddns name _acme-challenge.acme-www.pkgsrc.org. TXT; }; }; And last but not least, deployment of certificates via make, i.e. completely independent of acme.sh. Due to all of the above, acme.sh does not need to tentacle about in the filesystem and can run as a plain user in a chroot. It's not a tiny chroot, though (20M), since acme.sh needs a bunch of common shell tools: awk basename cat chmod cp curl cut date egrep/grep head mkdir mktemp mv nsupdate od openssl printf readlink rm sed sh sleep stat tail touch tr uname, and their shared libs, /libexec/ld.elf_so and /usr/libexec/ld.elf_so; under the chroot /etc a resolv.conf, the CA cert for Let's Encrypt (mozilla-rootcert-60.pem) and to make openssl complain less an empty openssl.cnf and in the chroot /dev: null, random and urandom. I call both the acme.sh --cron job and the certificate deployment make from daily.local, which adds the output to the daily mail and makes it easy to keep an eye on things. [Less]

Over the past month, I was coordinating and coding the remaining post-GSoC tasks. This mostly covers work around honggfuzz and sanitizers. honggfuzz ptrace(2) features I've introduced new ptrace(2) tests verifying attaching to a stopped process. ... [More] This is an important scenario in debuggers, the ability to call a ptrace(2) operation with the PT_ATTACH argument with a process id (PID) of a process entity that is stopped. In typical circumstances PT_ATTACH causes an executing process to stop and emit SIGSTOP for the tracer. An already stopped process is a special case as we cannot stop it again. Not every UNIX-like kernel can handle this scenario in a sensible way, and the modern solution is to keep the process stopped (rather than e.g... resumed) and emit a new signal SIGSTOP to the debugger (rather than e.g. not emitting anything). There used to be complex workarounds for mainstream kernels in debuggers such as GDB to workaround kernel bugs of attaching to a stopped process. honggfuzz is a security oriented, feedback-driven, evolutionary, easy-to-use fuzzer with interesting analysis options. This piece of software is developed by a Google employee, however the product is not an official Google software. honggfuzz uses on featured platforms ptrace(2) to monitor crash signals in traced processes. I've implemented a new backed in the fuzzer for NetBSD using its ptrace(2) API. The backend is designed to follow the existing scenarios and features in Linux & Android: Attaching to a manually stopped process, optionally spawned by the fuzzer. Option to attach to a selected process over its PID. Crash instruction decoding with aid of a disassembler (capstone for NetBSD). Ability to monitor multiple processes with an arbitrary number of threads. Monitor forks(2) and vforks(2) events, however unused in the current fuzzing model. Concurrent execution of multiple processes. Timeout of long-lasting (hanging?) processes in persistent mode (limit: 0.25[sec]). There are few missing features: Intel BTS - hardware assisted tracing Intel PT - hardware assisted tracing hp libunwind - unwinding stack of a traced process for better detection of unique crashes Sanitizers I've started researching Kernel Address Sanitizer, checking the runtime internals and differences between its version ABIs. My intention was to join efforts with Siddharth (GSoC student) and head with a sanitzier for EuroBSDCon 2018 in Romania. However, Maxime Villard decided to join the efforts a little bit earlier and he managed to get quickly a functional bare version for NetBSD/amd64. In the end we have decided to leave the kASan work to Maxime for now and let Siddharth to work on a kCov (SanitizerCoverage) device. SanCov is a feature of compilers, designed as an aid for fuzzers to ship interesting information from a fuzzing point of view of a number of function calls, comparisons, divisions etc. Successful userland fuzzers (such as AFL, honggfuzz) use this feature as an aid in determining of new code-paths. It's the same with a renowned kernel fuzzer - syzkaller. While, I'm helping Siddharth to port a kcov(4) device to NetBSD, I've switched to the remaining pending tasks in userland sanitizers. I've managed to switch the sanitizers from syscall(2) and __syscall(2) - indirect system call API - calls to libc routines. The approach of using an indirect generic interface didn't work well in the NetBSD case, as there is the need to handle multiple ABIs, Endians, CPU architectures, and the C language ABI is not a good choice to serialize and deserialize arbitrary function arguments with various types through a generic interface. The discussion on the rationale is perhaps not the proper place, and every low-level C developer is well aware of the problems. It's better to restrict the discussion to the statement that it's not possible (not trivial) to call in a portable way all the needed syscalls, without the aid of per-case auxiliary switches and macros. There are also some cases (such as pipe(2)) when is is not possible to express the system call semantics with syscall(2)/__syscall(2). I've switched these routines to use internal libc symbols when possible. In the remaining cases I've used a fallback to libc's versions of the routines, with aid of indirect function pointers. I'm trying to detect the addresses of real functions with dlsym(3) calls. In the result, I've switched all the uses of syscall(2) and __syscall(2) and observed no regressions in tests. I'm also in the process of deduplication of local patches to sanitizers. My current main focus is to finish switching syscall(2) and __syscall(2) to libc routines (patch pending upstream), introduce a new internal version of sysctl(3) that bypasses interceptors (partially merged upstream) and introduces new interceptors for sysctl(3) calls. This is a convoluted process in the internals with the goal to make the sanitizers more reliable across NetBSD targets and manage to sanitize less trivial examples such as rumpkernels. The RUMP code uses internally a modified and private versions of sysctl*() operations and we still must keep the internals in order and properly handle the RUMP code. Merged commits The NetBSD sources: Merge FreeBSD improvements to the man-page of timespec_get(3) Remove unused symbols from sys/sysctl.h Add a new ATF ptrace(2) test: child_attach_to_its_stopped_parent Add await_stopped() in t_ptrace_wait.h Add a new ATF test parent_attach_to_its_stopped_child Add a new ATF ptrace(2) test: tracer_attach_to_unrelated_stopped_process Drop a duplicate instruction line [libpthread] Mark kernel-asan as done (by maxv) TODO.sanitizers: Mark switch of syscall(2)/__syscall(2) to libc done The LLVM sources: Introduce new type for inteceptors UINTMAX_T Add internal_sysctl() used by FreeBSD, NetBSD, OpenBSD and MacOSX Improve portability of internal_sysctl() Try to fix internal_sysctl() for MacOSX Try to unbreak internal_sysctl() for MacOSX Summary I'm personally proud of the success of the reliability of the ptrace(2) backend in the renowned honggfuzz fuzzer. NetBSD was capable to handling all the needed features and support all of them with an issue-free manner. Once, I will address the remaining ptrace(2) issues on my TODO list - the NetBSD kernel will be capable to host more software in a similar fashion, and most importantly a fully featured debugger such as GDB and LLDB, however without the remaining hiccups. We are also approaching another milestone with the sanitizers' runtime available in the compiler toolchain: sanitizing rumpkernels. It is already possible to execute the rump code against a homegrown uUBSan runtime, but we are heading now to execute the code under the default runtime for the remaining sanitizers (ASan, MSan, TSan). For the record, it has been reported that kUBSan has been ported from NetBSD to at least two kernels: FreeBSD and XNU. Plan for the next milestone I'm in preparation for my visit to EuroBSDCon (Bucharest, Romania) in September and GSoC Mentor Summit & MeetBSDCa in October (California, the U.S.). I intend to rest during this month and still provide added value to the project, porting and researching missing software dedicated for developers. Among others, I'm planning to research the HP libunwind library and if possible, port it to NetBSD. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

The NetBSD Project is pleased to announce NetBSD 7.2, the second feature update of the NetBSD 7 release branch. It represents a selected subset of fixes deemed important for security or stability reasons, as well as new features and enhancements. If ... [More] you are running an earlier release of NetBSD and are (for whatever reasons) not able to update to the latest major release, NetBSD 8.0, we suggest updating to 7.2. For more details, please see the release notes. Complete source and binaries for NetBSD are available for download at many sites around the world and our CDN. A list of download sites providing FTP, AnonCVS, and other services may be found at the list of mirrors. [Less]