NetBSD

We are very happy to announce The NetBSD Foundation Google Summer of Code 2018 projects: Harsh Khatore - Modern cryptographic algorithms to netpgp, netpgpverify Nizar Benshaqi - SQL Database for ATF tests results with online query and ... [More] statistics page Marwa Desouky - Tickless Kernel with high-resolution timers Harry Pantazis - Kernel Undefined Behavior SANitizer Does025 - Porting FreeBSD Atheros driver to NetBSD Saad Mahmood - Machine-independent EFI bootloader Yang Zheng - Integrate libFuzzer With the Basesystem Keivan Motavalli - configuration files versioning in pkgsrc R3x - Implementing Kernel Address Sanitizer (KASan) in the NetBSD kernel We have started the community bonding period where students get in touch with mentors and community and get more confident with documentation and the code base. The coding period will start from May 14 until August 6. Please welcome all our students and a big good luck to students and mentors! A big thank you also to Google and The NetBSD Foundation Organization Administrators! Looking forward to a nice Google Summer of Code! [Less]

The NetBSD Project is pleased to announce NetBSD 8.0 RC 1, the first release candidate for the upcoming NetBSD 8.0 release. 25 years and a few days after the first official NetBSD release (NetBSD 0.8 on April 19, 1993) we are now quickly approaching ... [More] the first final release from the netbsd-8 branch that has been in the work for more most of a year now. The official RC1 announcement list these major changes compared to older releases: USB stack rework, USB3 support added In-kernel audio mixer Reproducible builds PaX MPROTECT (W^X) memory protection enforced by default on some architectures with fine-grained memory protection and suitable ELF formats: i386, amd64, evbarm, landisk, pmax PaX ASLR enabled by default on: i386, amd64, evbarm, landisk, pmax, sparc64 MKPIE (position independent executables) by default for userland on: i386, amd64, arm, m68k, mips, sh3, sparc64 added can(4), a socket layer for CAN busses added ipsecif(4) for route-based VPNs made part of the network stack MP-safe NET_MPSAFE kernel option is required to try WAPBL stability and performance improvements Specific to i386 and amd64 CPUs: Meltdown mitigation: SVS (separate virtual address spaces) Spectre mitigation (support in gcc, used by default for kernels) SMAP support (U)EFI bootloader Various new drivers: nvme(4) for modern solid state disks iwm(4), a driver for Intel Wireless devices (AC7260, AC7265, AC3160...) ixg(4): X540, X550 and newer device support. ixv(4): Intel 10G Ethernet virtual function driver. bta2dpd - new Bluetooth Advanced Audio Distribution Profile daemon Many evbarm kernels now use FDT (flat device tree) information (loadable at boot time from an external file) for device configuration, the number of kernels has decreased but the numer of boards has vastly increased. Lots of updates to 3rd party software included: GCC 5.5 with support for Address Sanitizer and Undefined Behavior Sanitizer GDB 7.12 GNU binutils 2.27 Clang/LLVM 3.8.1 OpenSSH 7.6 OpenSSL 1.0.2k mdocml 1.14.1 acpica 20170303 ntp 4.2.8p11-o dhcpcd 7.0.3 Lua 5.3.4 The NetBSD developers and the release engineering team have spent a lot of effort to make sure NetBSD 8.0 will be a superb release, but we have not yet fixed most of the accompanying documentation. So the included release notes and install documents will be updated before the final release, and also the above list of major items may lack important things. Get NetBSD 8.0 RC1 from our CDN (provided by fastly) or one of the ftp mirrors. Complete source and binaries for NetBSD are available for download at many sites around the world. A list of download sites providing FTP, AnonCVS, and other services may be found at http://www.NetBSD.org/mirrors/. Please test RC1, so we can make the final release the best one ever so far. We are looking forward to your feedback. Please send-pr any bugs or mail us at releng at NetBSD.org for more general comments. [Less]

I've spent a month on fixes and debugging issues around the tracing facilities in the kernel. Distribution cleanup As planned in the previous month, I've performed cleanup in the distribution: Removal of unneeded PT_SET_SIGMASK and ... [More] PT_GET_SIGMASK, followed by pullup to NetBSD-8. regnsub(3) and regasub(3) API change has been abandoned as it causes backward compatibility bootstrap breakage from older distributions. Altering it could require additional changes in libtre, but it has been abandoned too. At the end I've finally removed just the USE_LIBTRE build option, replacing the libc's regex(3) implementation with an alternative library from libtre/agrep, because it no longer builds as a regex-replacement and the upstream development stalled few years ago. Backport of _UC_MACHINE_FP() to NetBSD-8 has been finished, this means that we can use the same code in NetBSD-8 and NetBSD-current in the sources of sanitizers. Improvements in the ATF ptrace(2) tests I've performed the following operations in the ATF ptrace(2) and related tests, in the following commit order: Correct all ATF failures in t_ptrace_x86_wait.h (debug registers) ATF: Correct a race bug in attach2 (t_ptrace_wait*) ATF: Reenable attach2 in t_ptrace_wait* Add a new function in ATF t_ptrace_wait*: await_zombie_raw() ATF t_ptrace_wait*: Disable debug messages in msg.h ATF: Add new test race1 in t_ptrace_wait* Add new ATF tests: kernel/t_zombie Zombie detection race I've detected that the operation polling for a single process status - using sysctl(3) - can report the same process twice, for the first time as an alive process and for the second time as a zombie. This used to break the ATF ptrace(2) tests where we have a polling function, detecting the state change of a dying process. I've prepared an applied a fix for this case and the bug detected in ATF ptrace(2) tests is now gone. I've included additional regression tests as noted below, to catch the race in a dedicated t_zombie test-suite. There is also a controversy here, as it's not specified by POSIX what happens when we are polling an unrelated process, that is not a child of its parent. This operation can fail and we the previous approach we could report the same process twice, while with the newer one reporting twice is much more unlikely on the cost of missing the process at times. My preference is that a process between the state transition of alive->dying->dead and dead->zombie, can disappear for a while. I find this metaphor more natural rather than observing two entities that one is dying and the other is a zombie. There is no perfect solution to this process watch and just following the narrow cases requested with POSIX seems to be the proper solution and it de facto assumes such races. Bohrbug in X86 Debug Registers The release engineering machines were observing occasional failures with the X86 Debug Registers in the ptrace(2) ATF test-suite. This was appearing once a while, approximately quarterly, on both amd64 and i386 machines. I've missed them previously in my original work a year ago, as this happened to be caused by the fact that this bug is not reproducible on newer (quicker? more cores?) Intel CPUs. This failure was reproduced only in software-emulation (slow!) qemu and on Intel Core 2 Duo. An attempt to execute the same X86 Debug Register test on Intel i7 for 100M times does not make this crash to pop up even a single time. I've spent the majority of the past month on researching this bug and it happened to be a bohrbug, this means that adding almost any debug code anywhere makes it disappear completely or make reproducible siginificantly less often. It's not worth the space to describe the process of understanding this bug more closely, step by step - it's worth to mention the current state of the understanding of mine. We can observe a repeated syscall for the _lwp_kill(2) operation (executed with raise(SIGSTOP)), with the same trap frame (or very similar). I still don't know how is it possible. I've finally added support to the NetBSD kernel - in my local copy of the sources - for Bochs-style debug console/protocol (ISA port 0xe9). This allows me to log internal qemu debug and NetBSD kernel messages into a single buffer and output to a single file. I need this property as matching two logging buffers isn't trivial, especially since just the logging of interrupt frames in qemu can quickly generate hundreds of megabytes of text. Assuming that I've reproduced the bohrbug after 20k executions of the same test and there is a lot of noise from unrelated kernel/hardware traps, it's a useful property. I'm planning to cleanup this code and submit to the NetBSD sources. For completeness I had to patch the qemu isa-debugcon (Bochs-style debug console) source code to log into the same buffer as the internal qemu debug messages... sometimes bugs require non-conventional approaches to research them. Ideally I would like to have a rewind/record feature in qemu together with the gdb-server emulation, but we are still not there. ptrace(2) status and plans Now there is the X86 Debug Register race. Also, the following tests still marked as expected failures: eventmask3 resume1 signal3, signal5, signal6, signal7, signal8, signal9 suspend1, suspend2 vfork1 This shows that the remaining major problems are in: vfork(2) signals threads Once I will squash the X86 Debug Register race, I plan to work on these bugs in this oder: vfork handling bugs, signals and threads. I have in mind addition of a lot of a lot of tests verifying correct signal processing in traced programs. In this iteration of ptrace(2) hardening, I plan to skip Machine-Dependent extensions (like AVXv2 registers) and extending core(5) files with additional features (for example we might want to know exact ABI or FPU hardware layout). Plan for the next milestone Keep working on the bug reproducible with X86 Debug Registers and switch to the remaining ptrace(2) failures. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL, and chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

In recent years, no one in the releng team has had adequate time to devote to release engineering work. To address this shortcoming, the foundation has agreed to pay me part time to carry out the following duties: process pullup tickets write and ... [More] update release notes coordinate with the security team plan for future releases and release milestones identify and document critical bugs preventing a release create documents/web-/wikipages describing the current status of releases/release branches and keep them up to date identify missing pullups and push developers to submit pullup requests While this does not automatically provide developer time (still subject to volunteer time and motivation), it is however still a major step forward towards handling the release cycle, which is (currently) taking too long, in a professional manner. We hope to Speed up the release process for all pending releases on all active branches, as specified by the releng team and documented on the releng wiki page(s) Make the release process more transparent by providing up to date and easily accessible status documents Create better releases by making sure critical bugs get identified and fixed (if possible) To help the other issue mentioned above, we plan to offer (small) bug bounties for bugs identified as release show-stoppers. [Less]