Stjin, Here are a few random thoughts of mine -
Running a "Dining Philosophers" emulation in Jean J.Labrosse's uCos (or uC/os - II/III
<https://www.micrium.com/rtos/rtos-comparison/> <http://www.embedded.com/search?keyword=ucos>)
might qualify for what you want.
eCOS says it is an open-source real-time OS.
You might adapt the 8080 assembly language 'Kernel' by Max Agoston published in Dr. Dobb's Journal around 1977
<http://archive.6502.org/publications/dr_dobbs_journal/dr_dobbs_journal_vol_02.pdf>
A set of cooperating tasks in Forth might also qualify (& fit). There are FOSS Forth systems.
Any set of tasks that are interrupt driven could qualify as a 'custom' OS with 'real-time' performance.
Do you have to use certain concepts such as semaphores, messages, per-task/thread stack switching clock interrupts, etc.?
In any case, you will need to decide/justify what constitutes an 'OS', what 'running' involves, and what level of 'real-time' you need to demonstrate.
Andrew S. Tanenbaum's book "Operating Systems Design and Implementation" might help you answer those questions.
Fred
For my last year project i have to use a stm32l051 microcotroller. This controller has 8k RAM and 32K of flash. I have to run an RTOS on this system. But in my first research is looks difficult or even impossible to run a program with Mbed OS (rtos) features on a 8K RAM microcontroller.
What are your ideas about this topic and can you give me some advice?
Thank you very much.