oss-sec mailing list archives

From: Ali Polatel <alip () hexsys org> Date: Sat, 10 Jan 2026 18:54:39 +0000

Dear kind people,

I hope you're fine and healthy. The reason I am writing this mail is to share
a few of my experiments and research I've done to come up with a reasonable
stack pivot detection for the Syd kernel. TL;DR I have failed and I have learned
a lot. I have also learned everyone is doing various levels of wrong and I have
yet to come up with a "correct" solution. Are we doing it wrong or are we attacking
the wrong link? Please discuss.

The obvious idea is to check if stack pointer points to a valid stack region at
various boundaries. This solution is so obvious you'd imagine even a 4-year-old
can come up with it, yet G**gle has a patent[1] on it. Curious (mis)use of software
patents where an entity patents the equation "foo < bar < baz". What do we do if
they go one step ahead and patent "<"? Use emoji for maths? Move to Mars?

Anyhow, this is not what I am here to discuss. OpenBSD does something similar
with MAP_STACK[2] and Windows 8 had a similar mitigation until someone demonstrated
a trivial bypass[3]. Finding out this bypass was an important step forward for me so
I went ahead and rewrote it[4] for UNIX and verified it bypasses what Syd had
at the time and what OpenBSD has. This bypass is a simple improvement of what
OpenBSD regression tests have and involves a quick jumpback to the stack. Funnily
if you remove the printf in the intermediate stage of the bypass OpenBSD's SP check
at write(2) boundary will catch and kill you so keep it quiet as you bypass this
delicate mitigation ;).

Another roadblock for the SP points to a stack region detection is in userspace
there's no clear definition of "stack". The stack of the main thread is handled by
the kernel where everything is fine but thread stacks are typically your language
runtime's business. As an example if you check the stack of a Go thread on Linux,
you'll most probably find the VMA is named " Go: heap". It may be the stack for now,
but maybe it was not a bit ago or won't be a bit later. Asking Go devs to use
MAP_STACK would probably be rejected because it breaks the whole model of how they
do multithreading...

I digged a bit deeper and found what I thought was an improvement at the time.
Why not check the frame pointer instead of the stack pointer? Why not both?
It was fairly easy to patch Syd for this so I got to testing. This way of detection
is imho more reliable and less prone to bypassing however there's a big issue.
Both gcc and clang imply -fomit-frame-pointer with -O2 so you'll have a hard time
finding a binary in the wild these days that has frame pointers. If I were OpenBSD,
I'd compile the world with -fno-omit-frame-pointer and move on with my life and I'd
humbly and kindly recommend them to do that in short of any better ideas.

Now, my tests showed me another problem. Stack pivotting is not as unusual and as
malicious as you think. In fact, various programs make use of it in arguably weird
ways to achieve their goals. One example is Firefox's crashhelper, bash, gawk, ceph,
... I can easily come up with dozens more if I enable this mitigation and build Exherbo
packages under Syd as we proudly enable package testing by default. So even if you'd come
up with a reliable, efficient way to detect stack pivot, you're gonna have loads of false
positives to manage. Good luck.

Finally, after I ended my experiments and reverted[5] Syd's stack pivot detection. I
came across LKRG's README[6] incidentally which mentions validating the stack pointer
with "pCFI", their version of coarse-grained CFI, which also seems to check the
frame pointer[7] against the stack. I'd be curious to know whether there's any added
mechanism to detect stack pivot when the binary is compiled without frame pointers when
the frame pointer is reused by compiler for different purposes.

Best regards,
Ali Polatel

[1]: https://patents.google.com/patent/US10853480B2/en
[2]: https://isopenbsdsecu.re/mitigations/map_stack/
[3]: https://archive.ph/xS2Fl#selection-13.0-243.52
[4]: https://gitlab.exherbo.org/sydbox/sydbox/-/blob/main/dev/stackpivot-jumpback-bypass.c
[5]: https://gitlab.exherbo.org/sydbox/sydbox/-/commit/f03db6c677ddf5dbf87adeb6bd5efb0677869104
[6]: https://github.com/lkrg-org/lkrg/blob/b8b1418a6c1e7229cdf3dfa020fcc4945e108d83/README#L505
[7]:
https://github.com/lkrg-org/lkrg/blob/b8b1418a6c1e7229cdf3dfa020fcc4945e108d83/src/modules/exploit_detection/p_exploit_detection.c#L1585-L1591

Attachment: publickey - alip@hexsys.org - 0xC22DA9DE.asc Description:

Attachment: signature.asc Description: OpenPGP digital signature

Current thread:

• The Curious Case of Stack Pivot Detection Ali Polatel (Jan 10)