nanobsd: Use gpart and create code image before full disk image
The attached patch brings two main changes to the nanobsd script:
1- gpart is used instead of fdisk;
2- the code image is created first, and then used to ``assemble'' the
full disk image.
The patch was first proposed on the freebsd-embedded list:
http://lists.freebsd.org/pipermail/freebsd-embedded/2012-June/001580.html
and is currently under discussion:
http://lists.freebsd.org/pipermail/freebsd-embedded/2014-January/002216.html
Another effect is that the -f option ("suppress code slice extraction")
now imples the -i option ("suppress disk image build").
imp@ applied Patch by hand to new legacy.sh, plus tweaked for NANO_LOG vs
NANO_OBJ confusion in original.
Warner Losh [Thu, 15 Jul 2021 03:06:08 +0000 (21:06 -0600)]
loader: make sure CPUTYPE is ignored when building
CPUTYPE?=native causes -march=native to be added to the command
line. When the host machine is haswell, this causes some versions of
clang to generate code that can't execute in the efi boot loader
environment. Set _CPUCFLAGS= to undo what's done bsd.cpu.mk. bsd.cpu.mk
is included too early to control with NO_CPU_CFLAGS here. The only other
option is to put that in all the Makefiles, and this is less tedious and
error prone.
Alfonso Gregory [Wed, 14 Jul 2021 21:48:35 +0000 (15:48 -0600)]
Remove incorrect __restricted labels from strcspn
strcspn should never have had the __restrict keywords. While both of
these strings are const, it may have unindended side effects. While this
is the kernel, the POSIX definition also omits restrict.
These issues have low impact because they require precise circumstances
to trigger one of them. The disk must be > 2 TiB in size and either:
- The primary GPT header is dammaged.
- The freebsd-boot partiton is located farther than the first 2 TiB of
the disc and one of its sectors takes place at a lba value that makes
the higher 32 bits of this very value change.
Errors and corrections folow:
- decl and incl don't affect CF, so replace with subl/addl $1
- repe uses %cx, so move size to it with movw
- moving a 64-bit value with %cx of 2 (should be 4) so addresses
> 2TB will work.
PR: 233180
Reviewed by: imp@ (applied patch using description in bug)
Differential Revision: https://reviews.freebsd.org/D31100
Warner Losh [Tue, 13 Jul 2021 06:00:33 +0000 (00:00 -0600)]
cam_iosched: use tunable flag and make a bool really a bool
kern.cam.do_dynamic_iosched is really a bool, so change its type to
bool. While I'm here, also use the CTLFLAG_TUN flag instead of a
separate tunable line for it and kern.cam.iosched_alpha_bits.
Young Xiao [Tue, 21 May 2019 07:36:29 +0000 (15:36 +0800)]
Fix potential NULL pointer dereference of device physical path
In ata_dev_advinfo() and nvme_dev_advinfo(), if the physical path is
being stored and there is a malloc failure (malloc(9) is called with
M_NOWAIT), we could wind up in a situation where the device's
physpath_len is set to the length the user provided, but the physpath
itself is NULL.
If another context then comes in to fetch the physical path value, we
would wind up trying to memcpy a NULL pointer into the caller's buffer.
So, set the physpath_len to 0 when we free the physpath on entry into
the store case for the physical path. Reset the length to a non-zero
value only after we've successfully malloced a buffer to hold it.
This code mirrors scsi_xpt.c does already as well.
Signed-off-by: Young Xiao <92siuyang@gmail.com>
Reviewed by: imp
PR: 238014
John Hood [Sun, 11 Jul 2021 14:44:12 +0000 (08:44 -0600)]
loader: support.4th resets the read buffer incorrectly
Large nextboot.conf files (over 80 bytes) are not read correctly by the
Forth loader, causing file parsing to abort, and nextboot configuration
fails to apply.
Simple repro:
nextboot -e foo=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
shutdown -r now
That will cause the bug to cause a parse failure but shouldn't otherwise
affect the boot. Depending on your loader configuration, you may also
have to set beastie_disable and/or reduce the number of modules loaded
to see the error on a small console screen. 12.0 or CURRENT users will
also have to explicitly use the Forth loader instead of the Lua loader.
The error will look something like:
Warning: syntax error on file /boot/loader.conf.local
foo="xxxxxxxxxxxxxxnextboot_enable="YES"
^
/boot/support.4th has crude file I/O buffering, which uses a buffer
'read_buffer', defined to be 80 bytes by the 'read_buffer_size'
constant. The loader first tastes nextboot.conf, reading and parsing
the first line in it for nextboot_enable="YES". If this is true, then
it reopens the file and parses it like other loader .conf files.
Unfortunately, the file I/O buffering code does not fully reset the
buffer state in the reset_line_reading word. If the last file was read
to the end, that doesn't matter; the file buffer is treated as empty
anyway. But in the nextboot.conf case, the loader will not read to the
end of file if it is over 80 bytes, and the file buffer may be reused
when reading the next file. When the file is reread, the corrupt text
may cause file parsing to abort on bad syntax (if the corrupt line has
<>2 quotes in it), the wrong variable to be set, no variable to be set
at all, or (if the splice happens to land at a line ending) something
approximating normal operation.
The bug is very old, dating back to at least 2000 if not before, and is
still present in 12.0 and CURRENT r345863 (though it is now hidden by
the Lua loader by default).
Suggested one-line attached. This does change the behavior of the
reset_line_reading word, which is exported in the line-reading
dictionary (though the export is not documented in loader man pages).
But repo history shows it was probably exported for the PNP support
code, which was never included in the loader build, and was removed 5
months ago.
One thing that puzzles me: how has this bug gone unnoticed/unfixed for
nearly 2 decades? I find it hard to believe that nobody's tried to do
something interesting with nextboot, like load a kernel and filesystem,
which is what I'm doing.
Andriy Gapon [Mon, 21 Jun 2021 08:05:04 +0000 (11:05 +0300)]
rtwn: make sure to not write in upper bits of txdseq
ni_txseqs is kept as 16-bit counter, but we need to trim the upper four
bits as they may have special meanings for the firmware / hardware.
For instance, bit 15 enables hardware / firmware generation of sequence
numbers that overrides sequence numbers programmed by the driver.
Warner Losh [Thu, 8 Jul 2021 19:53:18 +0000 (13:53 -0600)]
devmatch: don't announce autoloading so much
devmatch rc script would announce it was loading a module multiple
times. It used kldload -n so it really wasn't loading it that many
times, but the message is confusing. Use kldstat to see if we need to
load the module before saying we do. This fixes the vast majority of the
problems. It may be possible to race devmatch with a user invocation and
devd, though quite hard. In that case we'll announce things twice, but
still only load it once. No attempt is made to fix this.
Warner Losh [Thu, 8 Jul 2021 19:44:21 +0000 (13:44 -0600)]
devmatch: Be tolerant of .ko being present.
We document that we did not need .ko on the module names in
devmatch_blocklist, but we really needed them. Keep the documentation
the same, but strip the .ko when we need to use the names so you can
specify either.
devmatch loads a number of things automatically. Allow the list of
things to load to happen first in case those drivers affect what would
be loaded. Normally, this will produce the same results, but there's
some special cases that may not when drivers are loaded that report
other drivers missing, like virtio_pci.
Warner Losh [Thu, 11 Mar 2021 15:42:09 +0000 (08:42 -0700)]
config_intrhook: provide config_intrhook_drain
config_intrhook_drain will remove the hook from the list as
config_intrhook_disestablish does if the hook hasn't been called. If it has,
config_intrhook_drain will wait for the hook to be disestablished in the normal
course (or expedited, it's up to the driver to decide how and when
to call config_intrhook_disestablish).
This is intended for removable devices that use config_intrhook and might be
attached early in boot, but that may be removed before the kernel can call the
config_intrhook or before it ends. To prevent all races, the detach routine will
need to call config_intrhook_train.
Sponsored by: Netflix, Inc
Reviewed by: jhb, mav, gde (in D29006 for man page)
Differential Revision: https://reviews.freebsd.org/D29005
Rather than allocating however much memory userspace asks for we only
allocate enough for a handful of states, and copy to userspace for each
completed row.
We start out with enough space for 16 states (per row), but grow that as
required. In most configurations we expect at most a handful of states
per row (more than that would have other negative effects on packet
processing performance).
Happily this wasn't a real bug, because pf_killstates() never fails, but
we should check the return value anyway, in case it does ever start
returning errors.
pidx is never NULL, and is used unconditionally later on in the
function.
Add an assertion, as documentation for the requirement to provide an idx
pointer.
Eugene Grosbein [Mon, 17 May 2021 21:03:15 +0000 (04:03 +0700)]
ipfw: reload sysctl.conf variables if needed
Currently ipfw has multiple components that are not parts
of GENERIC kernel like dummynet etc. They can bring in important
sysctls if enabled with rc.conf(5) and loaded with ipfw startup script
by means of "required_modules" after initial consult
with /etc/sysctl.conf at boot time. Here is an example of one
increasing limit for dummynet hold queues that defaults to 100:
net.inet.ip.dummynet.pipe_slot_limit=1000
This makes it possible to use ipfw/dummynet rules such as:
ipfw pipe 1 config bw 50Mbit/s queue 1000
Such rule is rejected unless above sysctl is applied.
Another example is a group of net.inet.ip.alias.* sysctls
created after libalias.ko loaded as dependency of ipfw_nat.
This is not a problem if corresponding code compiled in custom kernel
so sysctls exist when sysctl.conf is read early or kernel modules
loaded with a loader. This change makes it work also for GENERIC
and modules loaded by means of rc.conf(5) settings.
Eugene Grosbein [Wed, 19 May 2021 13:02:31 +0000 (20:02 +0700)]
rc.d: unbreak sysctl lastload
/etc/rc.d/securelevel is supposed to run /etc/rc.d/sysctl lastload
late at boot time to apply /etc/sysctl.conf settings that fail
to apply early. However, this does not work in default configuration
because of kern_securelevel_enable="NO" by default.
Add new script /etc/rc.d/sysctl_lastload that starts unconditionally.
Warner Losh [Fri, 16 Jul 2021 04:05:31 +0000 (22:05 -0600)]
UPDATING: Not unusual side effect of the awk bug fixed in 40a925385fa6b
You might not be able to build the kernel if you have an awk between Jul
10th and today. In stable/12, more (all) platforms than -current or
stable/13 have the first offset of the thread structure in the 0xfxx
range, which triggers this problem. If you've updated awk, you'll likely
need to follow the instructions here to install a fixed version before
you'll be able to build a kernel.
Warner Losh [Thu, 15 Jul 2021 22:46:06 +0000 (16:46 -0600)]
awk: revert upstream's attempt to disallow hex strings
Upstream one-true-awk decided to disallow hex strings as numbers. This
is in line with awk's behavior prior to C99, and allowed by the POSIX
standard. The standard, however, allows them to be treated as numbers
because that's what the standard said in the 2001 through 2004 editions.
Since 2001, the nawk in FreeBSD has treated them as numbers, so restore
that behavior, allowed by the standard.
A number of scripts in the FreeBSD tree depend on this interpretation,
including scripts to build the kernel which had mysteriously started
failing for some people and not others. By re-allowing 0x hex numbers,
this fixes those scripts and restores POLA.
Upstream issue: https://github.com/onetrueawk/awk/issues/126
Sponsored by: Netflix
Reviewed by: kevans
MFC After: asap due to regression alrady merged to stable
Differential Revision: https://reviews.freebsd.org/D31199
pf: allow table stats clearing and reading with ruleset rlock
Instead serialize against these operations with a dedicated lock.
Prior to the change, When pushing 17 mln pps of traffic, calling
DIOCRGETTSTATS in a loop would restrict throughput to about 7 mln. With
the change there is no slowdown.
Creating tables and zeroing their counters induces excessive IPIs (14
per table), which in turns kills single- and multi-threaded performance.
Work around the problem by extending per-CPU counters with a general
counter populated on "zeroing" requests -- it stores the currently found
sum. Then requests to report the current value are the sum of per-CPU
counters subtracted by the saved value.
Sample timings when loading a config with 100k tables on a 104-way box:
stock:
pfctl -f tables100000.conf 0.39s user 69.37s system 99% cpu 1:09.76 total
pfctl -f tables100000.conf 0.40s user 68.14s system 99% cpu 1:08.54 total
patched:
pfctl -f tables100000.conf 0.35s user 6.41s system 99% cpu 6.771 total
pfctl -f tables100000.conf 0.48s user 6.47s system 99% cpu 6.949 total
Stefan Eßer [Sat, 10 Jul 2021 11:00:56 +0000 (13:00 +0200)]
libalias: fix divide by zero causing panic
The packet_limit can fall to 0, leading to a divide by zero abort in
the "packets % packet_limit".
An possible solution would be to apply a lower limit of 1 after the
calculation of packet_limit, but since any number modulo 1 gives 0,
the more efficient solution is to skip the modulo operation for
packet_limit <= 1.
Michael Tuexen [Fri, 11 Jun 2021 18:14:34 +0000 (20:14 +0200)]
tcp: tolerate missing timestamps
Some TCP stacks negotiate TS support, but do not send TS at all
or not for keep-alive segments. Since this includes modern widely
deployed stacks, tolerate the violation of RFC 7323 per default.
Alexander Motin [Tue, 6 Jul 2021 02:19:48 +0000 (22:19 -0400)]
nvme(4): Report NPWA before NPWG as stripesize.
New Samsung 980 SSDs report Namespace Preferred Write Alignment of
8 (4KB) and Namespace Preferred Write Granularity of 32 (16KB).
My quick tests show that 16KB is a minimal sequential write size
when the SSD reaches peak IOPS, so writing much less is very slow.
But writing slightly less or slightly more does not change much,
so it seems not so much a size granularity as minimum I/O size.
Thinking about different stripesize consumers:
- Partition alignment should be based on NPWA by definition.
- ZFS ashift in part of forcing alignment of all I/Os should also
be based on NPWA. In part of forcing size granularity, if really
needed, it may be set to NPWG, but too big value can make ZFS too
space-inefficient, and the 16KB is actually the biggest supported
value there now.
- ZFS recordsize/volblocksize could potentially be tuned up toward
NPWG to work as I/O size granularity, but enabled compression makes
it too fuzzy. And those are normally user-configurable things.
- ZFS I/O aggregation code could definitely use Optimal Write Size
value and may be NPWG, but we don't have fields in GEOM now to report
the minimal and optimal I/O sizes, and even maximal is not reported
outside GEOM DISK to be used by ZFS.
Import the latest bsd-features branch of the one-true-awk upstream:
o Move to bison for $YACC
o Set close-on-exec flag for file and pipe redirects that aren't std*
o lots of little fixes to modernize ocde base
o free sval member before setting it
o fix a bug where a{0,3} could match aaaa
o pull in systime and strftime from NetBSD awk
o pull in fixes from {Net,Free,Open}BSD (normalized our code with them)
o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop)
Also revert a few of the trivial FreeBSD changes that were done slightly
differently in the upstreaming process. Also, our PR database may have
been mined by upstream for these fixes, and Mikolaj Golub may deserve
credit for some of the fixes in this update.
Alex Richardson [Mon, 21 Sep 2020 19:03:07 +0000 (19:03 +0000)]
awk: Fix subobject out-of-bounds access
When matching a regex with ^, it would attempt to access
gototab[NSTATES][NCHARS+2], and therefore access the state for the \002
character instead. This change is required to run awk under CHERI (with
sub-object bounds) and when running with UBSan instrumentation.
Warner Losh [Mon, 3 Jun 2019 05:25:22 +0000 (05:25 +0000)]
Another partial revert of r301289.
In this case, a change was made in one-true-awk from *FS to
getsval(fsloc) in a line just after one of the lines that had the 0 ->
NULL change. It works both ways as far as I can tell. It looks like a
bug fix, but I've not tried to track down which ancient version of
one-true-awk it was in (github starts too late for tracking this
down). Before and after the changes the regression suite is passes
100% relative to the un-modified one-true-awk.
Warner Losh [Mon, 3 Jun 2019 05:25:16 +0000 (05:25 +0000)]
Fix mismerge that crept into r301289.
The conversion of 0 -> NULL required a rebase at some point, as noted
in r301289 when pfg commited it. In that rebase, three lines remained
that had been removed in a prior version of awk, and one of them had a
0 -> NULL change causing a conflict. The conflict should have been
resolved by removing the three lines, but wasn't. This introduces a
regression into f.split3 test which prior to this commit we were
failing, but a pure onetrueawk wasn't. Remove the offending 3 lines.
Warner Losh [Sun, 2 Jun 2019 16:25:07 +0000 (16:25 +0000)]
Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git
Note: this backs out a number of changes we've made to awk because
they aren't upstream, but are on the vendor branch. Those will be
reapplied. svn makes it needlessly difficult to know which ones, but
at least r315426, r301289, and maybe r301691, though there may be
others too. None of these are critical, so bisecting through this
point is safe for all but awk regression tests :).
Chuck Tuffli [Sun, 27 Jun 2021 22:14:52 +0000 (15:14 -0700)]
bhyve: Fix NVMe iovec construction for large IOs
The UEFI driver included with Rocky Linux 8.4 uncovered an existing bug
in the NVMe emulation's construction of iovec's.
By default, NVMe data transfer operations use a scatter-gather list in
which all entries point to a fixed size memory region. For example, if
the Memory Page Size is 4KiB, a 2MiB IO requires 512 entries. Lists
themselves are also fixed size (default is 512 entries).
Because the list size is fixed, the last entry is special. If the IO
requires more than 512 entries, the last entry in the list contains the
address of the next list of entries. But if the IO requires exactly 512
entries, the last entry points to data.
The NVMe emulation missed this logic and unconditionally treated the
last entry as a pointer to the next list. Fix is to check if the
remaining data is greater than the page size before using the last entry
as a pointer to the next list.
Kristof Provost [Tue, 29 Jun 2021 08:26:40 +0000 (10:26 +0200)]
pf: Reduce the data returned in DIOCGETSTATESNV
This call is particularly slow due to the large amount of data it
returns. Remove all fields pfctl does not use. There is no functional
impact to pfctl, but it somewhat speeds up the call.
It might affect other (i.e. non-FreeBSD) code that uses the new
interface, but this call is very new, so there's unlikely to be any. No
releases contained the previous version, so we choose to live with the
ABI modification.
Kristof Provost [Mon, 28 Jun 2021 10:48:20 +0000 (12:48 +0200)]
pf tests: Stress state retrieval
Create and retrieve 20.000 states. There have been issues with nvlists
causing very slow state retrieval. We don't impose a specific limit on
the time required to retrieve the states, but do log it. In excessive
cases the Kyua timeout will fail this test.
Warner Losh [Fri, 21 Jun 2019 03:49:36 +0000 (03:49 +0000)]
Mount and unmount devfs around calls to add packages.
pkg now uses /dev/null for some of its operations. NanoBSD's packaging
stuff didn't mount that for the chroot it ran in, so any config that
added packages would see the error:
pkg: Cannot open /dev/null:No such file or directory
when trying to actually add those packages. It's easy enough for
nanobsd to mount /dev and it won't hurt anything that was already
working and may help things that weren't (like this). I moved the
mount/unmount pair to be in the right push/pop order from the
submitted patch.
PR: 238727
Submitted by: mike tancsa
Tested by: Karl Denninger
Warner Losh [Wed, 18 Nov 2020 19:22:24 +0000 (19:22 +0000)]
mergemaster: handle symbolic links during update.
/etc/os-release is now a symbolic link to a generated file. Make
mergemaster cope with symbolic links generically. I'm no longer
a big mergemaster user, so this has only been lightly tested
by me, though Kimura-san has ran it through its paces.
Olivier Houchard [Wed, 30 Jun 2021 20:56:50 +0000 (22:56 +0200)]
arm: Make sure we can handle a thumb entry point.
Similarly to what's been done on arm64 with commit 712c060c94fd447c91b0e6218c12a431206b487a, when executing a binary, if the
entry point is a thumb symbol, then make sure we set the PSL_T flag, otherwise
the CPU will interpret it in ARM mode, and that will likely leads to an
undefined instruction.
Ed Maste [Sun, 27 Jun 2021 17:21:26 +0000 (13:21 -0400)]
Add deprecation notice for WITH_PROFILE option
As discussed on freebsd-current [1] and freebsd-arch [2] and review
D30833, FreeBSD 14 will ship without the _p.a libraries built with -pg.
Both upstream and base system (in commit b762974cf4b9) Clang have been
modified to remove the special case for linking against these libraries.
Clang's -pg support and mcount() remain, so building with -pg can still
be used on code that the user builds; we just do not provide prebuilt
libraries compiled with -pg. A similar change is still needed for GCC.
Mariusz Zaborski [Fri, 11 Jun 2021 15:35:36 +0000 (17:35 +0200)]
libnv: optimize nvlist size calculation
If we had a multiple nvlist, during nvlist_pack, we calculated the size
of every nvlist separately. For example, if we had a nvlist with three
nodes each containing another (A contains B, and B contains C), we first
calculated the size of nvlist A (which contains B, C), then we calculate
the size of B (which contains C, notice that we already did the
calculation of B, when we calculate A), and finally C. This means that
this calculation was O(N!). This was done because each time we pack
nvlist, we have to put its size in the header
(the separate header for A, B, and C).
To not break the ABI and to reduce the complexity of nvlist_size,
instead of calculating the nvlist size when requested,
we track the size of each nvlist.
Mariusz Zaborski [Sun, 17 Feb 2019 18:26:27 +0000 (18:26 +0000)]
libnv: fix double free
In r343986 we introduced a double free. The structure was already
freed fixed in the r302966. This problem was introduced
because the GitHub version was out of sync with the FreeBSD one.
Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
MFC with: r343986
Mariusz Zaborski [Sun, 10 Feb 2019 23:28:55 +0000 (23:28 +0000)]
libnv: fix memory leaks
nvpair_create_stringv: free the temporary string; this fix affects
nvlist_add_stringf() and nvlist_add_stringv().
nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain
of nvpairs (as resetting it prevents nvlist_destroy() from freeing it).
Note: freeing the chain in nvlist_destroy() is not sufficient, because
it would still leak through nvlist_take_nvlist_array(). This affects
all nvlist_*_nvlist_array() use
Submitted by: Mindaugas Rasiukevicius <rmind@netbsd.org>
Reported by: clang/gcc ASAN
MFC after: 2 weeks
sbin/md5: improve compatibility with coreutils -c option
The -c option expects a digest file in either BSD or coreutils format.
The output for matched and mismatched files is identical to that
of the coreutils version.
The review of these changes included test cases that have already
been committed for the functionality that existed before.
Another test script is added to cover the coreutils compatible
extension implemented by this patch.
This commit contains a tests/Makefile that has been cleaned up
compared to the review version, using an implicit rule to apply the
TESTBASE path at build time (and the scripts have been renamed to
have an extension of .SH instead of .sh to trigger this rule).
md5: Improve compatibility with coreutils and format fix
The previous changes that added support for the coreutils -c option
modified the output generated by passing -r to match that of the
coreutils versions. The difference is that coreutils separates the
hash from the file name by two blanks " " (or a blank followed by
an asterisk " *" with the -b option denoting).
While most scripts or users will not notice the difference, it might
be considered a violation of POLA and this commit reverts the change
for the non-sum programs. These will print a single blank " " as the
separator, as they die before the previous commit.
In order to still generate output that is identical to that of the
coreutils programs, this commit generates the " " or " *" separator
used by them for the -sum versions, depending on the presence of the
-b option.
Warner Losh [Thu, 20 May 2021 17:26:46 +0000 (11:26 -0600)]
md5: portability fix -- include stdbool.h explicitly
stdbool.h needs to be included to use type bool variables. Due to
namespace pollution, this gets brought in on FreeBSD, but not on
other systems. Include it explicilty.
Warner Losh [Wed, 19 May 2021 17:26:20 +0000 (11:26 -0600)]
md5: Create md5sum, etc compatible programs
On Linux, there's a similar set of programs to ours, but that end in the
letters 'sum'. These act basically like FreeBSD versions run with the -r
option. Add code so that when the program ends in 'sum' you get the
linux -r behavior. This is enough to make most things that use sha*sum
work correctly (the -c / --check options, as well as the long args are
not implemented). When running with the -sum programs, ignore -t instead
of running internal speed tests and make -c an error.
Reviewed by: sef, and kp and allanjude (earlier version)
Relnotes: yes
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D30309
While the correctness of the supported hash algorithms can be tested
with the built-in self-test feature, these test cases are meant to
detect changes in the output format.
A follow-up commit will improve the compatibility with the coreutils
versions of the hash programs, and these tests should detect any
unintended side-effects of such a change on existing functionality.
sbin/md5: fix test case to not write outside temp directory
The bsd-p-test did create files 1.out .. 8.out in the test directory.
This has been fixed and the temporary output files are now written
to the temporary directory.
Lutz Donnerhacke [Thu, 27 May 2021 21:42:54 +0000 (23:42 +0200)]
libalias: Switch to SPLAY trees
Current data structure is using a hash of unordered lists. Those
unordered lists are quite efficient, because the least recently
inserted entries are most likely to be used again. In order to avoid
long search times in other cases, the lists are hashed into many
buckets. Unfortunatly a search for a miss needs an exhaustive
inspection and a careful definition of the hash.
Splay trees offer a similar feature - almost O(1) for access of the
least recently used entries), and amortized O(ln(n) - for almost all
other cases. Get rid of the hash.
Now the data structure should able to quickly react to external
packets without eating CPU cycles for breakfast, preventing a DoS.
Lutz Donnerhacke [Tue, 25 May 2021 13:03:34 +0000 (15:03 +0200)]
libalias: Restructure
Clean up the database handling in order to switch to more efficient
data structures. The development of this patch was artificially split
in to many small steps to ease reviewing.
- Common search terms
- Separate fully qualified search
- Separate table for partial links
- Cleanup _FindLinkIn
- Factor out the outgoing search function
- Factor out a common idiom to return found links
- Reorder incoming links by grouping of common search terms
- Remove LSNAT from outgoing search
- Group internal structure semantically
- Separate table for PPTP
- Use AliasRange instead of PORT_BASE
- Remove temporary state deleteAllLinks from global struct
- Avoid uninitialized expiration
Lutz Donnerhacke [Tue, 25 May 2021 07:07:21 +0000 (09:07 +0200)]
libalias: Promote per instance global variable timeStamp
Summary:
- Use LibAliasTime as a real global variable for central timekeeping.
- Reduce number of syscalls in user space considerably.
- Dynamically adjust the packet counters to match the second resolution.
- Only check the first few packets after a time increase for expiry.
Lutz Donnerhacke [Sat, 15 May 2021 15:35:36 +0000 (17:35 +0200)]
libalias: tidy up housekeeping
Replace current expensive, but sparsly called housekeeping
by a single, repetive action.
This is part of a larger restructure of libalias in order to switch to
more efficient data structures. The whole restructure process is
split into 15 reviews to ease reviewing. All those steps will be
squashed into a single commit for MFC in order to hide the
intermediate states from production systems.
As for example pfctl -ss keeps calling it, it saves a lot of overhead
from elided parsing of /etc/nsswitch.conf and /etc/protocols.
Sample result when running a pre-nvlist binary with nfs root and dumping
7 mln states:
before: 24.817u 62.993s 1:28.52 99.1%
after: 8.064u 1.117s 0:18.87 48.5%
projects / FreeBSD / FreeBSD.git / log