NetBSD

This report was written by Naveen Narayanan as part of Google Summer of Code 2019. This report encompasses the progress of the project during the third coding period. You can make sense of the overall progress of the project by ... [More] going through the first evaluation report and second evaluation report. WINE on amd64 Wine-4.4 (released on Mar 2019) is working fine on amd64 and i386. I have been able to use a script as a workaround for the problem of setting LD_LIBRARY_PATH. My patch for setting guard size to 0 and hence, precluding Wine from segfaulting, that got upstreamed, can be found here. I have updated the package to the latest development version of Wine which is Wine-4.13 (released on Aug 2019). I have added support to Wine pkgsrc packages to run tests using make test, and at the time of writing, they are failing. I have also noticed them fail on Linux non-pkgsrc environment and hence, will require further investigation. Initially, they were disabled owing to pkgsrc setting FORTIFY_SOURCE which is a macro that provides support for detecting buffer overflows. In pkgsrc, the wip/wine* packages honor PKGSRC_USE_FORTIFY variable passing _FORTIFY_SOURCE macro accordingly. Programs compiled with FORTIFY_SOURCE substitute wrappers for commonly used libc functions that don't do bounds checking regularly, but could in some cases. Wine unconditionally disables that via their configure script because for some platforms that triggered false positives in the past. However, in my experience, no false positive were found. Running tests on Wine-4.13 throws errors as you can see below: mode64# make test /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M adsldp.dll -p adsldp_test.exe.so sysinfo && touch sysinfo.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 002a:err:winediag:SECUR32_initNTLMSP ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution. 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f150,0x00000001,0x22f120) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EEA0 000000000002B028 000000000022EE9C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f0e0,0x00000001,0x22f0b0) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022EE30 000000000002B318 000000000022EE2C): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022F0C8: stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:advapi:LsaOpenPolicy ((null),0x22f980,0x00000001,0x22f950) stub 002a:fixme:security:GetWindowsAccountDomainSid (000000000022F6D0 000000000002B318 000000000022F6CC): semi-stub 002a:fixme:advapi:LsaClose (0xcafe) stub 002a:fixme:adsldp:sysinfo_get_UserName 0000000000021670,000000000022FB48: stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so cred && touch cred.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 002e:fixme:cred:CredReadW unhandled type -1 002e:fixme:cred:CredReadW unhandled flags 0xdeadbeef 002e:err:cred:CredWriteW bad username L"winetest" 002e:err:cred:CredWriteW bad username (null) 002e:err:cred:CredWriteW bad username (null) 002e:fixme:cred:CredDeleteW unhandled type -1 002e:fixme:cred:CredDeleteW unhandled flags 0xdeadbeef 002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub 002e:fixme:cred:CredReadDomainCredentialsW (0x1b1d0, 0x0, 0x22fa50, 0x22f908) stub 002e:fixme:cred:CredReadDomainCredentialsW (0x1b3c0, 0x0, 0x22fa50, 0x22f908) stub cred.c:820: Tests skipped: CRED_TYPE_DOMAIN_VISIBLE_PASSWORD credentials are not supported or are disabled. Skipping 002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked 002e:fixme:cred:CredIsMarshaledCredentialW BinaryBlobCredential not checked /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt && touch crypt.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_lmhash && touch crypt_lmhash.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md4 && touch crypt_md4.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_md5 && touch crypt_md5.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so crypt_sha && touch crypt_sha.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so eventlog && touch eventlog.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),(null)) stub 0046:fixme:advapi:OpenEventLogW (L"IDontExist",(null)) stub 0046:fixme:advapi:OpenEventLogW (L"IDontExist",L"deadbeef") stub 0046:fixme:advapi:OpenEventLogW Remote server not supported 0046:fixme:advapi:OpenEventLogW ((null),L"deadbeef") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW (L"",L"Application") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:GetEventLogInformation (0x0, 1, 0x0, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0x0, 0, 0x0, 0, 0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x0, 0, 0x22fb40) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x0) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 0, 0x22fb40) stub 0046:fixme:advapi:GetEventLogInformation (0xcafe4242, 0, 0x22fb59, 8, 0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x0) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x0) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0x0,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x0) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:GetOldestEventLogRecord (0x0,0x22fb40) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:BackupEventLogW (0x0,(null)) stub 0046:fixme:advapi:BackupEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,(null)) stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:BackupEventLogW (0x0,L"backup2.evt") stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),(null)) stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"idontexist.evt") stub 0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",(null)) stub 0046:fixme:advapi:OpenBackupEventLogW (L"IDontExist",L"idontexist.evt") stub 0046:fixme:advapi:OpenBackupEventLogW Remote server not supported 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub eventlog.c:546: Tests skipped: We don't have a backup eventlog to work with 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x0,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000000,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x0,0x0) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000000,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0x0,0x00000005,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000000,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000001,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000002,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000d,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x0000000e,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000007,0x00000000,0x1b3d0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa84) stub eventlog.c:479: Tests skipped: No records in the 'Application' log 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Application") stub 0046:fixme:advapi:BackupEventLogW (0xcafe4242,L"backup.evt") stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:OpenBackupEventLogW ((null),L"backup.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,L"backup2.evt") stub 0046:fixme:advapi:ClearEventLogW (0x0,(null)) stub 0046:fixme:advapi:CloseEventLog (0x0) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0020,0x0000,0x00000000,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:ReadEventLogA (0xcafe4242,0x00000005,0x00000000,0x1b1e0,0x00000038,0x22fa88,0x22fa8c) stub 0046:fixme:advapi:ClearEventLogW (0xcafe4242,(null)) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"Wine"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"Wine"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0004,0x0001,0x00000001,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000002,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc1"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc1"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0010,0x0001,0x00000003,0x0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc20") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0001,0x00000004,0x0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc300"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc300"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0002,0x0001,0x00000005,0x0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0000,0x0002,0x00000006,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0010,0x0002,0x00000007,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc1") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x7f7ff72beaf0,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0008,0x0002,0x00000008,0x1b3d0,0x0002,0x00000000,0x1b1e0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:RegisterEventSourceA ((null),"WineSrc20"): stub 0046:fixme:advapi:RegisterEventSourceW (L"",L"WineSrc20"): stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0002,0x0002,0x00000009,0x1b3d0,0x0000,0x00000000,0x0,0x0): stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:DeregisterEventSource (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"WineSrc300") stub 0046:fixme:advapi:ReportEventA (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x7f7ff72beb00,0x0): stub 0046:fixme:advapi:ReportEventW (0xcafe4242,0x0001,0x0002,0x0000000a,0x1b3d0,0x0001,0x00000000,0x1b1e0,0x0): stub 0046:err:eventlog:ReportEventW L"First string" 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:GetOldestEventLogRecord (0xcafe4242,0x22fa8c) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub 0046:fixme:advapi:OpenEventLogW ((null),L"Wine") stub 0046:fixme:advapi:GetNumberOfEventLogRecords (0xcafe4242,0x22fa80) stub 0046:fixme:advapi:CloseEventLog (0xcafe4242) stub eventlog.c:889: Tests skipped: No events were written to the eventlog 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "this name is too long", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x0) stub 0046:fixme:advapi:StartTraceA (0x0, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:StartTraceA (0x22fb40, "wine", 0x1b1e0) stub 0046:fixme:advapi:ControlTraceA (cafe4242, "wine", 0x1b1e0, 1) stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so lsa && touch lsa.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 004a:fixme:advapi:LsaOpenPolicy ((null),0x22f960,0x000f0fff,0x22f930) stub 004a:fixme:security:GetWindowsAccountDomainSid (000000000022F690 0000000000019078 000000000022F68C): semi-stub 004a:fixme:advapi:LsaEnumerateAccountRights (0xcafe,0x22fa20,0x22f990,0x22f92c) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb10,0x000f0fff,0x22fae8) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb00) stub 004a:fixme:advapi:LsaClose (0xcafe) stub 004a:fixme:advapi:LsaOpenPolicy ((null),0x22fb40,0x00000800,0x22fb08) stub /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/tools/runtest -q -P wine -T ../../.. -M advapi32.dll -p advapi32_test.exe.so registry && touch registry.ok /usr/pkg/emul/netbsd32/lib/wine/libwine.so.1: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/ntdll.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernel32.dll.so: text relocations /usr/pkg/emul/netbsd32/lib/wine/wine/kernelbase.dll.so: text relocations 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. 004e:fixme:reg:RegQueryInfoKeyA security argument not supported. 004e:fixme:reg:RegQueryInfoKeyW security argument not supported. registry.c:2850: Tests skipped: HKCR key merging not supported registry.c:3146: Tests skipped: HKCR key merging not supported 004e:fixme:winspool:PerfOpen (null): stub 004e:fixme:winspool:PerfCollect L"Global", 0x22ead8, 0x22eabc, 0x22eac0: stub 004e:fixme:winspool:PerfClose stub 004e:fixme:winspool:PerfOpen (null): stub 004e:fixme:winspool:PerfCollect L"invalid counter name", 0x22ead8, 0x22eabc, 0x22eac0: stub 004e:fixme:winspool:PerfClose stub registry.c:4032: Test failed: [ 9] expected 1168, got 2 registry.c:4032: Test failed: [10] expected 1814, got 2 *** Error code 2 Stop. make[1]: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64/dlls/advapi32/tests *** Error code 1 Stop. make: stopped in /usr/pkgsrc/wip/wine64-dev/work/wine-4.13/wine64 Running programs on Wine You can find obligatory screenshots of Wine-4.4/4.13 on amd64/i386 below: 010 Editor (Professional Text/Hex Editor) on Wine-4.13 (amd64) Notepad++ on Wine-4.4 (i386) Pinball on Wine-4.13 (amd64) How to run Wine on NetBSD/amd64 Compile kernel with USER_LDT enabled. Install kernel. Clone wip repo. cd /usr/pkgsrc/wip/wine64; make install (for 4.4) or cd /usr/pkgsrc/wip/wine64-dev; make install (for 4.13) wine notepad Future Plans Wine requires the kernel option USER_LDT to be able to run 32-bit applications on amd64 - facilitated by WoW64. Presently, this feature isn't enabled by default on NetBSD and hence, the kernel has to be compiled with USER_LDT enabled. I will work on getting the tests to pass and finding a better approach to deal with LD_LIBRARY_PATH issue. In addition to that, I shall work on incorporating a NetBSD VM to Wine Test Bot infrastructure so we can preclude Wine from getting out of shape on NetBSD in the future (work in progress). Summary Presently, Wine-4.4 and Wine-4.13 are working fine on amd64/i386. I have been able to meet all the goals as per my original plan. I would like to attribute the success to the support provided by my mentors @leot, @maya and @christos. I would also like to thank my mentor @maxv for solving the conflict between SVS and USER_LDT kernel options. I intend to maintain the packages wine64/wine64-dev/wine32 for the foreseeable future. As stated above, I shall try to set up a NetBSD VM as Wine Test Bot for purpose of CI. Once again, thanks to Google for enabling me to do what I love. Source Code Commit: Wine Commit: pkgsrc WIP Package: Wine64 Package: Wine64-dev Package: Wine32 [Less]

Once upon a time a developer wrote a temperature sensor driver for one system and decided to port it to another, similar, system. For days and then weeks, the developer waited and waited for the other, similar, system to arrive. One day, joy of ... [More] joys, the other, similar, system arrived. A National Day of Celebration commenced, and the porting effort began. Over hours, perhaps even as many as five hours of time, the sensors were finally able to tell you whether they were hot or not. This other, similar, system suddenly was purposeless and was pondering what else life could provide when the Remote Server task popped up and said "Hey, first media file is free", and sadly, this other, similar, system tried its first media file, and then purchased many, many more media files, and suddenly this other, similar, system was suddenly hooked. Unfortunately, this other, similar, system had a problem talking on the network without falling asleep on the job, so the developer says "let's try a USB network instead!", and initially this seemed like a good idea. Many bits were transferred over USB, but soon whispers of a lurking monster, known to developers, experience or otherwise, as KASSERT, were heard and soon found common. The developer attempted other USB network as the other, similar, system was destined to be flown many thousands of miles away soon, but the only other option was similarly plagued by the KASSERT monster. The developer reached into his Bag Of Holding and pulled out a magical weapon known capable of slaying the KASSERT monster. The mighty blade of MPSAFE was free again! After much planning and many failed attacks, the developer was able to slay the KASSERT monster and all the bits wanting to be transferred were able to be. For a day and for a night there were celebrations. Much food and ale was consumed until finally everyone was asleep, and the music and lights were finally turned off. In the morning a great clean up was needed and while the developer was cleaning off the shiny, happy and working USB network the original USB network was accidentally reconnected. Oh no, the KASSERT monster has returned! Lock your doors and hide your children. The developer quickly pulled out MPSAFE again, and once again the KASSERT monster was slain, though the face of this monster was different to the previous monster. The developer then searched and searched for surely they were more KASSERT monsters to be found. Indeed, many many others were found, though they retreated to safety after two more of their number were slain by the mighty MPSAFE. The developer called upon his friends Shared and Code and with them forged a new weapon against the KASSERT monster, using the mighty MPSAFE in ways unheard of before. After much research and planning, and with the help of some friends, the USBNET was born. All the angels and all the birds of the world were said to sing all at once at this moment, for the USBNET would bring happiness to both the Code Deletionist and the Code Sharers, bound to war against each other from time immemorial. With this new USBNET the developer was able to blaze a trail across the landscape, searching out each KASSERT monster lurking in every USB network corner. All told, fourteen faces of KASSERT monster were found and the developer and his friends have slain seven of these faces, with the remaining seven under attack, life looks grim for them. The other, similar, system is safe now. Turns out that MPSAFE also has cleared up the sleeping problem using the cousins, NET and FDT in a tight, dual-blade combination. Let the world rejoice, for soon the KASSERT monster will be no more! --mrg @ 2019-08-11 tl;dr: i fixed many bugs across several USB ethernet adapters and got sick of fixing the same bug across multiple drivers so made common code for them to use instead. the original 4 drivers fixed were axen(4), axe(4), cdce(4), and ure(4). the current list of fixed drivers, at time of writing, includes smsc(4), udav(4) and urndis(4). all drivers except umb(4) are ported but either not tested or not yet working with the usbnet(9) framework. update 2019-09-02: all 13 known working drivers converted and will be present in netbsd 9. [Less]

Once upon a time a developer wrote a temperature sensor driver for one system and decided to port it to another, similar, system. For days and then weeks, the developer waited and waited for the other, similar, system to arrive. One day, joy of ... [More] joys, the other, similar, system arrived. A National Day of Celebration commenced, and the porting effort began. Over hours, perhaps even as many as five hours of time, the sensors were finally able to tell you whether they were hot or not. This other, similar, system suddenly was purposeless and was pondering what else life could provide when the Remote Server task popped up and said "Hey, first media file is free", and sadly, this other, similar, system tried its first media file, and then purchased many, many more media files, and suddenly this other, similar, system was suddenly hooked. Unfortunately, this other, similar, system had a problem talking on the network without falling asleep on the job, so the developer says "let's try a USB network instead!", and initially this seemed like a good idea. Many bits were transferred over USB, but soon whispers of a lurking monster, known to developers, experience or otherwise, as KASSERT, were heard and soon found common. The developer attempted other USB network as the other, similar, system was destined to be flown many thousands of miles away soon, but the only other option was similarly plagued by the KASSERT monster. The developer reached into his Bag Of Holding and pulled out a magical weapon known capable of slaying the KASSERT monster. The mighty blade of MPSAFE was free again! After much planning and many failed attacks, the developer was able to slay the KASSERT monster and all the bits wanting to be transferred were able to be. For a day and for a night there were celebrations. Much food and ale was consumed until finally everyone was asleep, and the music and lights were finally turned off. In the morning a great clean up was needed and while the developer was cleaning off the shiny, happy and working USB network the original USB network was accidentally reconnected. Oh no, the KASSERT monster has returned! Lock your doors and hide your children. The developer quickly pulled out MPSAFE again, and once again the KASSERT monster was slain, though the face of this monster was different to the previous monster. The developer then searched and searched for surely they were more KASSERT monsters to be found. Indeed, many many others were found, though they retreated to safety after two more of their number were slain by the mighty MPSAFE. The developer called upon his friends Shared and Code and with them forged a new weapon against the KASSERT monster, using the mighty MPSAFE in ways unheard of before. After much research and planning, and with the help of some friends, the USBNET was born. All the angels and all the birds of the world were said to sing all at once at this moment, for the USBNET would bring happiness to both the Code Deletionist and the Code Sharers, bound to war against each other from time immemorial. With this new USBNET the developer was able to blaze a trail across the landscape, searching out each KASSERT monster lurking in every USB network corner. All told, fourteen faces of KASSERT monster were found and the developer and his friends have slain seven of these faces, with the remaining seven under attack, life looks grim for them. The other, similar, system is safe now. Turns out that MPSAFE also has cleared up the sleeping problem using the cousins, NET and FDT in a tight, dual-blade combination. Let the world rejoice, for soon the KASSERT monster will be no more! --mrg @ 2019-08-11 tl;dr: i fixed many bugs across several USB ethernet adapters and got sick of fixing the same bug across multiple drivers so made common code for them to use instead. the original 4 drivers fixed were axen(4), axe(4), cdce(4), and ure(4). the current list of fixed drivers, at time of writing, includes smsc(4), udav(4) and urndis(4). all drivers except umb(4) are ported but either not tested or not yet working with the usbnet(9) framework. [Less]

A number of the remaining reported ptrace(2) bugs are GDB related. The previous support for GDB in NetBSD was in need for refreshment, as it had no support for gdbserver capabilities. The GDB Server is an execution mode of the debugger, which spawns ... [More] a dedicated process that interacts with its tracee. The process then establishes a link (socket, serial, ...) with the GDB client that is controlled by a programmer. As NetBSD-9 has finally branched and I keep receiving requests to finish the integration of LLVM sanitizers, I have pushed this task forward too. I have also completed a few leftover tasks from my previous months that still needed fixes. GNU GDB Originally the GDB debugger was meant for local debugging. However with the request of tracing remote setups, it has been extended to remote debugging capabilities and this move standardized the protocol for many debuggers. In fact the native / local / non-remote debugging is redundant with the remote capabilities, however this blog entry is not the right place to discuss the technical merits in comparision between them trying to convince someone. What matters: the developers of LLDB (LLVM debugger) already removed their local debugging variation (for Linux) and implements only the remote process plugin. The NetBSD support for LLDB started with this mode of copying the Linux approach. Bypassing no longer exists for Linux local-debugging-only support plugin. Inside the GNU world, the transition from local process debugging to remote process debugging is progressing slowly. In the current state of affairs there is a need to support two separate (however whenever possible, with code sharing) plugins for each OS. Also, there is intention from some GNU debugger developers to completely drop the native-only process plugin to debugging code. The state in NetBSD one month ago handled only local-debugging and no code for remote process debugging. The state is the same with other BSD derived systems (FreeBSD, DragonFlyBSD...) and this state was in general kept alive with a reduced set of features compared to Linux. I have decided to refresh the GDB support for NetBSD as this is still the primary debugger on NetBSD (LLDB support is still work-in-progress) for two primary reasons: Slight ptrace(2) changes in the kernel code can alter support in GDB/NetBSD. With proper NetBSD handling in GDB and ability to run its regression tests the fallout will be minimized. Certain ptrace(2) reported bugs are in fact specific GDB bugs. In order to close them as resolved I need to revamp and validate the GDB support. I have managed to get a basic GDB client/server session to work for tracing a simple application. The following features have been implemented targetting as of now NetBSD/amd64: creating inferior (debuggee in GDB terminology) attaching killing detaching resuming (with optional: single step, signal, resuming single thread) waiting checking whether a thread is alive fetching registers storing registers reading memory writing memory requesting interruption auxv reading software breakpoint support hardware single step support svr4 ELF loader tracing retrieving exec file name from pid reading thread name What is still missing in gdbserver for NetBSD/amd64: i386 support multilib support amd64/i386 reading TLS BASE (x86) register (there is still missing a ptrace(2) interface) EXEC events handling improvement in the generic code (that maps Linux restricted semantics only) hardware watchpoint/breakpoint support FPU support FORK, VFORK, POSIX_SPAWN events reading loadmap research for other features: multiprocess, mulifs, async, btrace, tracepoints, new gdb connections.. The main difficulty was with overly elaborated version of Linux proccess tracing in GDB that contains various workaround for various kernel bugs, handles differences between behavior of the Linux kernel (like different set of ptrace(2) calls or.. swapped arguments) between CPUs.. Finally, I have decided to base my work on unfortunately dated (and probably broken today) gdbserver for lynxos/i386 and then keep adding missing features that are implemented for Linux. I have passed most of the past month on debugging spurious crashes and anomalies in my GDB remote port to NetBSD. After much work, I have finally managed to get gdbserver to work and perform successful operations of spawning a process, inserting a breakpoint, single-stepping, detaching and running a process until its natural completion. I plan to validate the native tracing support for NetBSD, checking whether it needs upgrading and planning to run the GDB regression tests as soon as possible. LLVM changes I have updated the list of ioctl(2) operations to the state as of 9.99.3 (+36 ioctls). The support for NVMM ioctl(2)s has been enabled and restricted as of now to amd64 only. I have also finally managed to resolve the crash of dynamic (as DSO library) Address Sanitizer that blocked the integration with base in Januray 2019. My original suspect on the root cause was finally verified to be false (mismatch in the source code files composing .so libraries). The real reason to ship with broken dynamic ASan was inability to reuse the same implementation of TSD (Thread-Specific Data) in asan-dynamic. The static (default in LLVM) version of ASan can use in early initialization TLS (Thread-Local Storage), while TSD (Thread-Specific Data) is broken. It happened that the state with dynamic ASan is inversed and TLS breaks and TSD works. I have also landed build rules for LLVM sanitizers into src/, for: asan ubsan tsan msan xray safestack libfuzzer These features are targetting in all the supported variations NetBSD/amd64. The build rules are still not hooked into the MKLLVM=yes build as I intend to backport the needed patches to llvm-8 enhancements from llvm-HEAD and then reimport that snapshot into the NetBSD's distribution, avoiding downstream patches. MAXRSS changes During the work on libfuzzer last year for GSoC-2018 one of the changes that landed the kernel was the restoration of the MAXRSS option. MAXRSS presents the maxiumum resident set size of the process during its lifetime. In order to finalize this work, there was still need to enable printing it in ps(1). I have pushed a kernel fix to update the RSS related values for sysctl(3) query of a process statistics and with help of Krzysztof Lasocki reenabled printing MAXRSS, IDRSS (integral unshared data), ISRSS (integral unshared stack) and IXRSS (integral shared memory size) in ps(1). These values were commented out in the ps(1) program since the inception of NetBSD, the first registered commit. $ ps -O maxrss,idrss,isrss,ixrss|head -3 PID MAXRSS IDRSS ISRSS IXRSS TTY STAT TIME COMMAND 910 5844 20 12 216 pts/0 Is 0:00.01 -ksh 1141 5844 20 12 152 pts/0 I+ 0:00.01 mg event.h These change have been backported to the NetBSD-9 branch and will land 9.0. Welcome back! Plan for the next milestone Start executing GDB regression tests. Keep enhancing GDB support. Keep detecting ptrace(2) bugs and addressing them. This work was sponsored by The NetBSD Foundation. The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can: http://netbsd.org/donations/#how-to-donate [Less]

This report was written by Maciej Grochowski as a part of developing the AFL+KCOV project. Recently I started working on Fuzzing Filesystems on NetBSD using AFL. You can take a look at the previous post to learn more details about background of ... [More] this project. This post summarizes the work that has been done in this area, and is divided into 3 sections: Porting AFL kernel mode to work with NetBSD Running kernel fuzzing benchmark Example howto fuzzing particular Filesystem AFL Port for NetBSD AFL is a well known fuzzer for user space programs and libraries, but with some changes it can be used for fuzzing the kernel binary itself. For the first step to fuzz the NetBSD kernel via AFL, I needed to modify it to use coverage data provided by the kernel instead of compiled instrumentations. My initial plan was to replace the coverage data gathered via afl-as with that provided by kcov(4). In this scenario, AFL would just run a wrapper and see the real coverage from the kernel. I also saw previous work done by Oracle in this area, where instead of running the wrapper as a binary, the wrapper code was included in a custom library (.so object). Both approaches have some pros and cons. One thing that convinced me to use a solution based on the shared library with initialization code was the potentially easier integration with remote fork server. AFL has some constraints in the way of managing fuzzed binary, and keeping it on a remote VM is less portable than fuzzing using a shared library and avoiding introducing changes to the original binary fuzzing. Porting AFL kernel fuzzing mode to be compatible with NetBSD kernel mainly relied on how the operating system manages the coverage data. The port can be found currently on github. Writing a kernel fuzzing benchmark Performance is one of the key factors of fuzzing. If performance of the fuzzing process is not good enough, it's likely that the entire solution won't be useful in practice. In this section we will evaluate our fuzzer with a practice benchmark. One exercise that I want to perform to check the AFL kernel fuzzing in practice is similar to a password cracking benchmark. The high level idea is that a fuzzer based on coverage should be much smarter than bruteforce or random generation. To do this, we can write a simple program that will take a text input and compare it with a hardcoded value. If the values match, then the fuzzer cracked the password. Otherwise, it will perform another iteration with a modified input. Instead of "password cracker", I called my kernel program "lottery dev". It's a character device that takes an input and compares it with a string. The chance to find one 6 byte combination (or the "lucky byte" combination, because of the name) are similar to winning big in the lottery: every byte contains 8 bits, thus we have 2**(8*6) => 281,474,976,710,656 combinations. The coverage based fuzzer should be able to do this much quicker and in fewer iterations, as feedback from code instrumentations will show, compared to blindly guessing. I performed a similar test using a simple C program: the program read stdio and compared it with the hardcoded pattern. If the pattern matches, the program panics, and otherwise returns zero. Such a test took an AFL about a few hours on my local laptop to break the challenge (some important details can make it faster). The curious reader who wants to learn some basics of AFL should try to do run a similar test on their machine. I ran the fuzzer on my lottery dev for several days, and after almost a week it was still not able to find the combination. So something was fundamentally not right. The kernel module with the wrapper code can be found here. Measuring Coverage for a particular function In the previous article, I mentioned that the NetBSD kernel seems to be 'more verbose' in terms of coverage reporting. I ran my lottery dev wrapper code (the code that writes a given input to the char device) to check the coverage data using standard kcov(4) without the AFL module. My idea was to check the ratio between entries of my code that I wanted to track and other kernel functions that can be considered noise from other subsystems. These operations are caused by the context services, such as Memory Management, File Systems or Power Management, that are executed in the same process. To my surprise, there was a lot of data, but I could not find any of functions from lottery dev. I quickly noticed that the amount of addresses is equal to the size of kcov(4) buffer, so maybe my data didn't fit in the buffer inside kernel space? I changed the size of the coverage buffer to make it significantly larger and recompiled the kernel. With this change I reran the test. Now, with the buffer being large enough, I collected the data and printed top 20 entries with a number of occurrences. There were 30578 entries in total. 1544 /usr/netbsd/src/sys/uvm/uvm_page.c:847 1536 /usr/netbsd/src/sys/uvm/uvm_page.c:869 1536 /usr/netbsd/src/sys/uvm/uvm_page.c:890 1536 /usr/netbsd/src/sys/uvm/uvm_page.c:880 1536 /usr/netbsd/src/sys/uvm/uvm_page.c:858 1281 /usr/netbsd/src/sys/arch/amd64/compile/obj/GENERIC/./machine/cpu.h:70 1281 /usr/netbsd/src/sys/arch/amd64/compile/obj/GENERIC/./machine/cpu.h:71 478 /usr/netbsd/src/sys/kern/kern_mutex.c:840 456 /usr/netbsd/src/sys/arch/x86/x86/pmap.c:3046 438 /usr/netbsd/src/sys/kern/kern_mutex.c:837 438 /usr/netbsd/src/sys/kern/kern_mutex.c:835 398 /usr/netbsd/src/sys/kern/kern_mutex.c:838 383 /usr/netbsd/src/sys/uvm/uvm_page.c:186 308 /usr/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/rb.c:129 307 /usr/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/rb.c:130 307 /usr/netbsd/src/sys/uvm/uvm_page.c:178 307 /usr/netbsd/src/sys/uvm/uvm_page.c:1568 231 /usr/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/rb.c:135 230 /usr/netbsd/src/sys/uvm/uvm_page.c:1567 228 /usr/netbsd/src/sys/kern/kern_synch.c:416 It should not be a surprise that the coverage data does not much help our fuzzing with AFL. Most of the information that the fuzzer sees is related to UVM page management and machine-dependent code. I decided to remove instrumentation from these most common functions to check the difference. Using the attribute no_instrument_function tells the compiler to not put instrumentation for coverage tracing inside these functions. Unfortunately, after recompiling the kernel, the most common functions did not disappear from the list. As I figured out, the support in GCC 7 may not be fully in place. GCC 8 for help To solve this issue, I decided to work on reusing GCC 8 for building the NetBSD kernel. After fixing basic build warnings, I got my basic kernel working. This still needs more work to get kcov(4) fully functional. Hopefully, in the next report, I will be able to share these results. Fuzzing Filesystem Given what we already know, we can run Filesystem fuzzing. As a target I chose FFS as it is a default FS that is delivered with NetBSD. The reader may ask the question: why would you run coverage based fuzzer if the data is not 100% accurate? So here is the trick: For coverage based fuzzers, it is usually recommended to leave the input format as is, as genetic algorithms can do a pretty good job here. There is great post on Michal Zalewski's Blog about this process applied for the JPEG format: "Pulling JPEGs out of thin air". But what will AFL do if we provide an input format that is already correct? We already know what a valid FS image should look like (or we can simply just generate one). As it turns out, AFL will start performing operations on the input in a similar way as mutation fuzzers do - another great source that explains this process can be found here: "Binary fuzzing strategies: what works, what doesn't" Writing mount wrapper As we discussed in the previous paragraph, to fuzz the kernel itself, we need some code to run operations inside the kernel. We will call it a wrapper, as it wraps the operations of every cycle of fuzzing. The first step to write a wrapper for AFL is to describe it as a sequence of operations. Bash style scripting is usually good enough to do that. We need to have an input that will be modified by the fuzzer, and be able to mount it. NetBSD comes with vnd(4) that allows exposing regular files as block devices. The simplest sequence can be described as: # Expose file from tmpfs as block device vndconfig vnd0 /tmp/rand.tmp # Create a new FS image on the blk dev that we created newfs /dev/vnd0 # Mount our fresh FS mount /dev/vnd0 /mnt # Check if FS works fine echo "FFS mounted!" > /mnt/test # Undo mount umount /mnt # Last undo step vndconfig -u vnd0 From bash to C and system calls At this point, the reader has probably figured out that a shell script won't be the best approach for fuzzer usage. We need to change it to C code and use proper syscall/libc interfaces. vndconfig uses the opendisk(3) combined with vnd_ioctl. mount(2) is a simple system call which can operate directly after file is added to vnd(4) Below is an example conceptual code for mounting an FS: // Structure required by mount() struct ufs_args ufs_args; // VNConfigure step rv = run_config(VND_CONFIG, dev, fpath); if (rv) printf("VND_CONFIG failed: rv: %d\n", rv); // Mount FS if (mount(FS_TYPE, fs_name, mntflags, &ufs_args, 8) == -1) { printf("Mount failed: %s", strerror(errno)); } else { // Here FS is mounted // We can perform any other operations on it // Umount FS if (unmount(fs_name, 0) == -1) printf("#: Umount failed!\n"); } // VNC-unconfigure rv = run_config(VND_UNCONFIG, dev, fpath); if (rv) { printf("VND_UNCONFIG failed: rv: %d\n", rv); } The complete code can be found here Ready to fuzz FFS! aka Running FS Fuzzing with a predifined corpus The first thing that we need to do is to have a wrapper that provides mount/umount functionality. In the previous section, we have already shown how that can be done. For now, we will be fuzzing the same kernel that we are running on. Isn't that dangerous? Taking a saw to the branch we are sitting on? Of course it is! In this exercise I want to illustrate an idea from a technical perspective so the curious reader is able to understand it better and do any modifications by their own. The take away from this exercise is that the fuzzing target is the kernel itself, the same binary that is running the fuzzing process. Let's come back to the wrapper code. We've already discussed how it works. Now we need to compile it as a shared library. This is not obvious, but should be easy to understand after we already brought this sawing-off metaphor. To compile the so object: gcc -fPIC -lutil -g -shared ./wrapper_mount.c -o wrapper_mount.so Now we need to create the input corpus, for the first attempt we will use a large enough empty file. dd if=/dev/zero of=./in/test1 bs=10k count=8 And finally run. The @@ tells AFL to put here the name of input file that will be used for fuzzing. ./afl-fuzz -k -i ./in -o ./out -- /mypath/wrapper_mount.so @@ Now, as we described earlier, we need a proper FS image to allow AFL perform mutations on it. The only difference is the additional newfs(8) command. # We need a file, big enough to fit FS image but not too big dd if=/dev/zero of=./in/test1 bs=10k count=8 # A block is already inside fuzzer ./in vndconfig vnd0 ./in/test1 # Create new FFS filesystem newfs /dev/vnd0 vndconfig -u vnd0 Now we are ready for another run! ./afl-fuzz -k -i ./in -o ./out -- /mypath/wrapper_mount.so @@ american fuzzy lop 2.35b (wrapper_mount.so) ┌─ process timing ─────────────────────────────────────┬─ overall results ─────┐ │ run time : 0 days, 0 hrs, 0 min, 17 sec │ cycles done : 0 │ │ last new path : none seen yet │ total paths : 1 │ │ last uniq crash : none seen yet │ uniq crashes : 0 │ │ last uniq hang : none seen yet │ uniq hangs : 0 │ ├─ cycle progress ────────────────────┬─ map coverage ─┴───────────────────────┤ │ now processing : 0 (0.00%) │ map density : 17.28% / 17.31% │ │ paths timed out : 0 (0.00%) │ count coverage : 3.53 bits/tuple │ ├─ stage progress ────────────────────┼─ findings in depth ────────────────────┤ │ now trying : trim 512/512 │ favored paths : 1 (100.00%) │ │ stage execs : 15/160 (9.38%) │ new edges on : 1 (100.00%) │ │ total execs : 202 │ total crashes : 0 (0 unique) │ │ exec speed : 47.74/sec (slow!) │ total hangs : 0 (0 unique) │ ├─ fuzzing strategy yields ───────────┴───────────────┬─ path geometry ────────┤ │ bit flips : 0/0, 0/0, 0/0 │ levels : 1 │ │ byte flips : 0/0, 0/0, 0/0 │ pending : 1 │ │ arithmetics : 0/0, 0/0, 0/0 │ pend fav : 1 │ │ known ints : 0/0, 0/0, 0/0 │ own finds : 0 │ │ dictionary : 0/0, 0/0, 0/0 │ imported : n/a │ │ havoc : 0/0, 0/0 │ stability : 23.66% │ │ trim : n/a, n/a ├────────────────────────┘ ┴─────────────────────────────────────────────────────┘ [cpu: 0%] Future work Support for the NetBSD kernel fuzzing was developed as a part of the AFL FileSystems Fuzzing project, which aims to improve quality of filesystems and catch various issues. The very next thing that I have on my todo list is to provide support for kernel tracing on GCC 8 to turn off coverage data from other functions that generate a lot of noise. During the FFS fuzzing, I found a few issues that I need to analyze in detail. Last but not least, for the next report I plan to show a remote setup of AFL running on a VM, reporting crashes, and being remotely rebooted by the master controller. [Less]

This report was prepared by Manikishan Ghantasala as a part of Google Summer of Code 2019 This report encloses the progress of the project Add KNF (NetBSD style) clang-format configuration during the second coding period of GSoC 2019. Clang-format ... [More] Clang-format is a powerful code formatter which is a part of clang. Clang-format formats the code either by a configuration file .clang-format or can be chosen from some predefined coding styles namely LLVM, Google, Chromium, Mozilla, WebKit. The final goal of the project is to add a new style NetBSD along with them by patching the libFormat to support the missing styles and add the configuration according to NetBSD KNF. clang-format -style=NetBSD Style options introduced in the first coding period: BitFieldDeclarationsOnePerLine SortNetBSDIncludes You can also take a look at the first report to learn more about these style options. Work done in the second coding period I have worked on the following styles during this second coding period. Withdrawn SortNetBSDIncludes and modified existing sortIncludes. Modified spacesBeforeTralingComments to support block comments. New style option alignConsecutiveListElements introduced. Sortincludes: The native SortIncludes sorts the includes/headers in alphabetical order. And we also have IncludeCategories which allows setting custom priorities to group the includes matching via a regex after sorting. Example: Configuration: IncludeCategories: -Regex: ^<(a|b|c)/ Priority: 1 -Regex: ^<(foo)/ Priority: 2 -Regex: .* Priority: 3 Input #include exe.h #include gaz.h #include #include #include #include #include #include #include Output #include #include #include #include #include #include #include #include gaz.h> Modified SortIncludes The new sortIncludes supports to give custom priority for sorting in addition to grouping priority. Now the IncludeCategories have a new field named SortPriority along with Priority to set the priority while sorting the includes, and the default priority will be alphabetical. The usage and working of the new IncludeCategories style shown in the following example. Example IncludeCategories: -Regex: <^c/ Priority: 1 SortPriority: 0 -Regex: ^<(a|b)/ Priority: 1 SortPriority: 1 -Regex: ^<(foo)/ Priority: 2 -Regex: .* Priority: 3 Input #include exe.h #include #include #include #include #include #include #include Output #include #include #include #include #include #include #include #include gaz.h As we observe in the above example, the includes having the same Priority are grouped, and SortPriority defines the sort priority. The patch was accepted and ready to merge, and you can find the patch here -> SortIncludesPatch. This patch also introduces the NetBSD style to clang-format with configurations to supported styles. Spaces Before Trailing Comments This option is also a native style option in clang-format which enables a user to decide the number of spaces to be given before trailing comments (// - comments) but not block comments (/* - comments). The reason for this is that block comments have different usage patterns and different exceptional cases. The following is an example of the native style option. SpacesBeforeTrailingComments: 3 void f() { if (true) { //foo f(); //bar } //foo1 } Modifications to spaceBeforeTrailingComments: I am working on modifying this style option to support block comments by covering cases where block comments are used. There are some cases yet to be covered. Once the patch is ready, this is the deliverable The initial revision can be found here -> patch SpacesBeforeTrailingComments: 3 void f() { if (true) { /*foo */ f(); /*bar */ } /*foo1*/ } AlignConsecutiveListElements AlignConsecutiveListElements is a new style option that I am going to introduce to clang-format. This style option aligns elements in consecutive definitions, declarations of lists. The style is not yet ready to patch, and I have to modify a lot of functionalities in alignTokens() which aligns tokens to support this style. Example: Input keys[] = { {"all", f_all, 0 }, { "cbreak", f_cbreak, F_OFFOK }, {"cols", f_columns, F_NEEDARG }, { "columns", f_columns, F_NEEDARG }, }; Output keys[] = { { "all", f_all, 0 }, { "cbreak", _cbreak, F_OFFOK }, { "cols", f_columns, F_NEEDARG }, { "columns", f_columns, F_NEEDARG }, }; The blocker to this style is the nested structure for list declarations, alignTokens() should be equipped to parse the nested list declarations to support this style option. I will make sure this style option is available by the next report. Further plans For the next phase, I will make all the style options that were modified or introduced till now during the first two phases mergeable to upstream along with required unit tests. With these style options ready, I will test the new clang-format with NetBSD style patched across NetBSD source and check for bugs. After the testing will try to fix the bugs and get the NetBSDStyle ready for the final evaluation. Summary In the final coding period, the main focus will be on testing the new/modified style options and fix them. Add any missing styles for NetBSD and get the NetBSDStyle by the final evaluation. I want to thank my mentors Michal, Christos and developers of both LLVM and NetBSD for supporting to complete this project. [Less]

This report was prepared by Jason High as a part of Google Summer of Code 2019 Introduction As a memory hard hashing scheme, Argon2 attempts to maximize utilization over multiple compute units, providing a defense against both Time Memory ... [More] Trade-off (TMTO) and side-channel attacks. In our first post, we introduced our GSOC project's phase 1 to integrate the Argon2 reference implementation into NetBSD. Having successfully completed phase 1, here we briefly discuss parameter tuning as it relates to password management and performance. Parameter Tuning Both the reference paper [1] and the forthcoming RFC [2] provide recommendations on how to determine appropriate parameter values. While there are no hard-and-fast rules, the general idea is to maximize resource utilization while keeping performance, measured in execution run-time, within a tolerable bound. We summarize this process as follows Determine the Argon2 variant to use Determine the appropriate salt length Determine the appropriate tag length Determine the acceptable time cost Determine the maximum amount of memory to utilize Determine the appropriate degree of parallelism Step 1 All three Argon2 variants are available in NetBSD. First, argon2i is a slower variant using data-independent memory access suitable for password hashing and password-based key derivation. Second, argon2d is a faster variant using data-dependent memory access, but is only suitable for application with no threats from side-channel attacks. Lastly, argon2id runs argon2i on the first half of memory passes and argon2d for the remaining passes. If you are unsure of which variant to use, it is recommended that you use argon2id.[1][2] Step 2-3 Our current implementation uses a constant 32-byte hash length (defined in crypt-argon2.c) and a 16-byte salt length (defined in pw_gensalt.c). Both of these values are on the high-end of the recommendations. Steps 4-6 We paramaterize Argon2 on the remaining three variables: time (t), memory (m), and parallelism (p). Time t is defined as the amount of required computation and is specified as the number of iterations. Memory m is defined as the amount of memory utilized, specified in Kilobytes (KB). Parallelism p defines the number of independent threads. Taken together, these three parameters form the knobs with which Argon2 may be tuned. Recommended Default Parameters For default values, [2] recommends both argon2id and argon2d be used with a time t cost of 1 and memory m set to the maximum available memory. For argon2i, it is recommended to use a time t cost of 3 for all reasonable memory sizes (reasonable is not well-defined). Parallelism p is a factor of workload partitioning and is typically recommended to use twice the number of cores dedicated to hashing. Evaluation and Results Given the above recommendations, we evaluate Argon2 based on execution run-time. Targeting password hashing, we use an execution time cost of 500ms to 1sec. This is slightly higher than common recommendations, but is largely a factor of user tolerance. Our test machine has a 4-core i5-2500 CPU @ 3.30GHz running NetBSD 8. To evaluate the run-time performance of Argon2 on your system, you may use either the argon2(1) or libargon2. Header argon2.h provides sufficient documentation, as well as the PHC git repo at [4]. For those wanting to use argon2(1), an example of using argon2(1) is below m2# echo -n 'password'|argon2 somesalt -id -p 2 -t 1 -m 19 Type: Argon2id Iterations: 1 Memory: 524288 KiB Parallelism: 2 Hash: 7b9618bf35b02c00cfef32cb4455206dc400b140116710a6c02732e068021609 Encoded: $argon2id$v=19$m=524288,t=1,p=2$c29tZXNhbHQ$e5YYvzWwLADP7zLLRFUgbcQAsUARZxCmwCcy4GgCFgk 0.950 seconds Verification ok In order to establish a performance baseline, we first evaluate the run-time of all three variants using the recommended default parameters without parallelism. Our objective is to maximize memory m while constraining the execution cost between 500ms to 1sec. While we graph all three variants for comparison, our target is variant argon2id. We loop over Argon2 with monotonically increasing values of m for all three variants. Graphing our results below, we determine the maximum memory m value within our bound is 219. However, to follow common suggestions [3], we chose 220 (1048576 Bytes) with the assumption that increased parallelism will bring the execution time cost down within acceptable limits. Figure 1 Having established our performance baseline to obtain a best guess for our memory m value, we can turn our attention to parallelism. As touched upon earlier, common suggestions for parallelism p is twice the dedicated core count. Threads in Argon2 form computational lanes which allow work on the memory matrix to be partitioned into independent slices. As such, intuition tells us that by increasing parallelism we should see a decrease in our overall run-time until we are computationally bound. In the graph below, we see the affects of increasing parallelism p over our selected t and m values. Figure 2 Our baseline memory m value initially exceed our desired upper bound of 1sec without parallelism. Fortunately, we found that increasing the thread count sufficiently parallelizes the work until it starts to settle around p=8. To see if we can further increase our baseline memory m value, we can look at following the graph for argon2id with t=1, p=8. We note that our initial baseline value of m=20 is the maximum m value falling within our bound. Figure 3 Having all three parameters, we can validate the run-time execution easily using argon2(1) to confirm performance is within our defined bounds. Once you are satisfied, you may use passwd.conf(5) to apply the parameters globally or on a per-user basis. Our first post includes an example of adding the appropriate stanza to passwd.conf(5). m2# echo -n 'password'|argon2 somesalt -id -m 20 -p 8 -t 1 Type: Argon2id Iterations: 1 Memory: 524288 KiB Parallelism: 8 Hash: c62dbebec4a2da3a37dcfa2d82bd2f55541fce80992cd2c1cb887910e859589f Encoded: $argon2id$v=19$m=524288,t=1,p=8$c29tZXNhbHQ$xi2+vsSi2jo33Potgr0vVVQfzoCZLNLBy4h5EOhZWJ8 0.858 seconds Verification ok Summary Argon2 is well-known for its high resource requirements. Fortunately, we are able to tune Argon2 so that we can achieve reasonable performance on NetBSD. In this post, we consider an approach to selecting appropriate parameter values for tuning Argon2 for password hashing. There are no hard-and-fast rules. We have found that it is easiest to start with a baseline value of memory m, then tweaking the values of parallelism p in order to achieve the desired performance. Remaining Tasks For the third and final portion of Google Summer of Code, we will focus on clean-up, documentation, and finalizing all imported code and build scripts. If you have any questions and/or comments, please let us know. References [1] https://github.com/P-H-C/phc-winner-argon2/blob/master/argon2-specs.pdf [2] https://tools.ietf.org/html/draft-irtf-cfrg-argon2-04#section-4 [3] https://argon2-cffi.readthedocs.io/en/stable/parameters.html [4] https://github.com/P-H-C/phc-winner-argon2 [Less]

Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest ... [More] in extending it to more languages. In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support, extending NetBSD's ptrace interface to cover more register types and fix compat32 issues, and lately fixing watchpoint support. You can read more about that in my June 2019 report. My July's work has been focused on improving support for NetBSD threads in LLDB. This involved a lot of debugging and fighting hanging tests, and I have decided to delay committing the results until I manage to provide fixes for all the immediate issues. Buildbot updates During July, upstream has made two breaking changes to the build system: Automatic switching to libc++abi when present in ebuild tree was initially removed in D63883. I needed to force it explicitly because of this. However, upstream has eventually reverted the change. LLDB has enabled Python 3 support, and started requiring SWIG 2+ in the process (D64782). We had to upgrade SWIG on the build host, and eventually switched to Python 3 as well. As a result of earlier watchpoint fixes, a number of new tests started running. Due to lacking multithreading support, I had to XFAIL a number of LLDB tests in r365338. A few days later, upstream has fixed the issue causing TestFormattersSBAPI to fail. I un-XFAILED it in r365991. The breaking xfer:libraries-svr4:read change has been reapplied and broke NetBSD process plugin again. And I've reapplied my earlier fix as r366889. Lit maintainers have broken NetBSD support in tests by starting to use env -u VAR syntax in r366980. The -u switch is not specified by POSIX, and not supported by NetBSD env(1). In order to fix the problem, I've changed FileCheck's behavior to consider empty envvar as equivalent to disabled (r367122), and then switched lit to set both envvars to empty instead (r367123). Finally, I've investigated a number of new test failures by the end of the month: New functionalities/signal/handle-abrt test was added in r366580. Since trampolines are not properly supported at the moment, I've marked it XFAIL in r367228. Two functionalities/exec tests started failing since upstream fixed @skipIfSanitized that previously caused the test to be skipped unconditionally (r366903). Since it's not a regression, I've marked it XFAIL in r367228. Same happened for one of the python_api/hello_world tests. It was clearly related to another failing test, so I've marked it XFAIL in r367285. Two tools/lldb-vscode tests were failing since upstream compared realpath'd path with normal path, and our build path happens to include symlinks as one of the parent directories. I've fixed the test to compare realpath in r367291. While at it, I've replaced weird os.path.split(...)[0] with clearer os.path.dirname(...) as suggested by Pavel Labath, in r367290. NetBSD ptrace() interfaces for thread control NetBSD currently provides two different methods for thread-related operations. The legacy method consists of the following requests: PT_CONTINUE with negative data argument. It is used to resume execution of a single thread while suspending all other threads. PT_STEP with positive data argument. It is used to single-step the specified thread, while all other threads continue execution. PT_STEP with negative data argument. It is used to single-step the specified thread, while all other threads remain suspended. This means that using those methods, you can effectively either: run all threads, and optionally send signal to the process, run one thread, while keeping other threads suspended, single-step one thread, with all other threads either running or being suspended as a whole. Furthermore, it is impossible to combine single-stepping with syscall tracing via PT_SYSCALL. The new method introduced by Kamil Rytarowski during his ptrace(2) work is more flexible, and includes the following requests: PT_RESUME that sets the specified thread to continue running after PT_CONTINUE. PT_SUSPEND that sets the specified thread to remain suspended after PT_CONTINUE. PT_SETSTEP that enables single-stepping for the specified thread after PT_CONTINUE. PT_CLEARSTEP that disables single-stepping for the specified thread. Using the new API, it is possible to control both execution and single- stepping per thread, and to combine syscall tracing with that. It is also possible to deliver a single signal either to the whole process or to one of the threads. Implementing threading in LLDB NetBSD process plugin When I started my work, the support for threads in the NetBSD plugin was minimal. Technically, the code had structures needed to keep the threads and filled it in at start. However, it did not register new or terminated threads, and it did not support per-thread execution control. The first change necessary was therefore to implement support for reporting new and terminated threads. I've prepared an initial patch in D65555. With this patch enabled, the thread list command now correctly reports the list of threads at any moment. The second change necessary is to fix process resuming routine to support multiple threads properly. The routine is passed a data structure containing requested action for each thread. The old code simply took the action for the first thread, and applied it to the whole process. D64647 is my work-in-progress attempt at using the new ptrace calls to apply correct action for each thread. However, the patch is currently buggy as it assumed that LLDB should provide explicit eStateSuspended action for each thread that is supposed to be supposed. The current LLDB implementation, on the other hand, assumes that thread should be suspended if no action is specified for it. I am currently discussing with upstream whether the current approach is correct, or should be changed to the explicit eStateSuspended usage. The third change necessary is that we need to explicitly copy debug registers to newly created threads, in order to enable watchpoints on them. However, I haven't gotten to writing a patch for this yet. Fixing nasty process interrupt bug While debugging my threading code, I've hit a nasty bug in LLDB. After issuing process interrupt command from remote LLDB session, the server terminated. After putting a lot of effort into debugging why the server terminates with no obvious error, I've discovered that it's terminating because… the client has disconnected. Further investigation with help of Pavel Labath uncovered that the client is silently disconnecting because it expects a packet indicating that the process has stopped and times out waiting for it. In order to make the server send this packet, NetBSD process plugin needed to explicitly mark process as stopped in the SIGSTOP handler. I've fixed it in r367047. Future plans The initial 6 months of my LLDB contract have passed. I am currently taking a month's break from the work, then I will resume it for 3 more months. During that time, I will continue working on threading support and my remaining goals. The remaining TODO items are: Add support to backtrace through signal trampoline and extend the support to libexecinfo, unwind implementations (LLVM, nongnu). Examine adding CFI support to interfaces that need it to provide more stable backtraces (both kernel and userland). Add support for i386 and aarch64 targets. Stabilize LLDB and address breaking tests from the test suite. Merge LLDB with the base system (under LLVM-style distribution). This work is sponsored by The NetBSD Foundation The NetBSD Foundation is a non-profit organization and welcomes any donations to help us continue funding projects and services to the open-source community. Please consider visiting the following URL to chip in what you can: https://netbsd.org/donations/#how-to-donate [Less]

Prepared by Siddharth Muralee(@R3x) as a part of Google Summer of Code’19 As a part of Google Summer of Code’19, I am working on improving the support for Syzkaller kernel fuzzer. Syzkaller is an unsupervised coverage-guided kernel fuzzer, that ... [More] supports a variety of operating systems including NetBSD. This report details the work done during the second coding period. You can also take a look at the first report to learn more about the initial support that we added. Network Packet Injection As a part of improving the fuzzing support for the NetBSD kernel. We decided to add support for fuzzing the Network stack. This feature already exists for operating systems such as Linux and OpenBSD. Motivation The aim is to fuzz the Network stack by sending packets with malformed/random data and see if it causes any kernel anomalies. We aim to send packets in such a way that the code paths in the kernel which would usually get triggered normally (during ordinary usage of the networking system) would also get triggered here. This is achieved using the TAP/TUN interface. TAP/TUN Interface TAP/TUN interface is a software-only interface - which means there are no hardware links involved. This makes it an ideal option to be used for interacting with the kernel networking code. Userspace programs can create TAP/TUN interfaces and then write properly formatted data into it which will then be sent to the kernel. Also reading from the TAP/TUN interface would give us the data that the interface is sending out. Basic Design We create a virtual interface using TAP/TUN and send packets through it. We add two syscalls - syz_emit_ethernet and syz_extract_tcp_res. The former does the job of sending a packet into the kernel and the latter receives the response. We need the response from the kernel to be read because we need the TCP acknowledgement number to be used for the next packet that we send. So syz_extract_tcp_res also extracts the acknowledgement number from the reply the kernel sent. This article explains the concept of TCP acknowledgement and sequence numbers very well. Parallelizing Network fuzzing In the syzkaller config you can define the number of processes that syzkaller can run in a single VM instance. And since we can have multiple instances of the fuzzer(executor) running at the same time we need to make sure that there is no collision between the interfaces. For solving this we create a seperate interface per fuzzer and assign it with a different IP address (both IPv4 and IPv6) to create an isolated network. Filesystem Image fuzzing A very less explored area in fuzzing is Filesystem image fuzzing. Syzkaller supports filesystem fuzzing only for the Linux kernel. We are currently working on porting the existing support that Linux has and then improve it. Motivation The aim is to fuzz filesystem specific code by mounting custom images and then perform operations on these images. This would lead to execution of the kernel components of filesystem code and allow us to find out potential bugs. Existing Design As I mentioned in the previous report - syzkaller generats inputs based on a pseudo formal grammar. This allows us to also write grammar to generate filesystem images on the fly. This is what the current implementation does - generate random images, write the segments into memory and then mount the resulting image. Miscellaneous work I have also been fine tuning the syzkaller fuzzer whenever necessary. This involves adding new features and fixing issues. Coverage Display Syzkaller has a utility which uses the coverage from KCoV and marks the corresponding lines on the source code and displays the same. This feature wasn’t working for some of the operating systems. The issue was that syzkaller was trying to strip off the common prefix on all the file paths retrieved from Kcov and then appending the directory where the source code is present to access the files. Stripping the common prefix made sense for Linux since its files were distributed across multiple folders in the src/ directory. This created issues for NetBSD since almost all of the kernel files are present in src/sys/ which lead to `sys` also being removed from the resultant path leading to an invalid file name. I worked on revamping the syz-manager such that it removes prefix computation altogether and take the prefix we need to strip as a part of the config for syz-manager. Then we add the filepath of the kernel sources to get the path to the file. The coverage for NetBSD can be viewed on the NetBSD dashboard. TODO The Filesystem fuzzing code isn’t upstream as of now and will be done shortly. I have added a basic support for fuzzing both the Filesystem and the Network stack. Now there are a lot of improvements to be done for the same. This mainly involves adding more detailed and improved descriptions. Relevant Links Syzkaller Dashboard for NetBSD Syzkaller repository on Github NetBSD docs on setting up syzkaller Summary So far, we have found around 70 unique crashes with syzkaller. During the final coding period - I would be working on improving the support for the Filesystem fuzzing. Last but not least, I want to thank my mentors, @kamil and @cryo for their useful suggestions and guidance. I would also like to thank Dmitry Vyukov, Google for helping with any issues faced with regard to Syzkaller. Finally, thanks to Google to give me a good chance to work with NetBSD community. [Less]