Kevin Day [Sat, 22 Nov 2025 22:59:52 +0000 (16:59 -0600)]
Update: Next micro version (0.7.4).
The following are the commands that I ran to make this change:
# find data/ sources/ specifications/ documents/ licenses/ install.sh -type f -exec sed -i -e 's|0\.7\.3|0.7.4|g' -e 's|^version_micro 3|version_micro 4|g' '{}' ';'
# find sources/ -name *.h -exec sed -i -e 's|_program_version_micro_s F_string_ascii_3_s|_program_version_micro_s F_string_ascii_4_s|g' -e 's|_program_version_micro_s_length F_string_ascii_3_s_length|_program_version_micro_s_length F_string_ascii_4_s_length|g' '{}' ';'
# sed -i -e 's|version_micro 3|version_micro 4|g' data/build/settings* data/build/stand_alone/settings.*
Kevin Day [Sat, 22 Nov 2025 14:22:59 +0000 (08:22 -0600)]
Update: Reduce the number of lock requests to check if process is interrupted.
The general rule is to check each arbitrarily large loop or between long running functions for interrupt.
Each lock attempt is also checked for interrupt before locking.
This is resulting in a bit too many checks (which are to some extend expensive because checking requires a lock).
Many of these checks are back to back because some explicit checks are called before normal locking requests (which themselves perform explicit checks).
Kevin Day [Fri, 21 Nov 2025 04:48:26 +0000 (22:48 -0600)]
Progress: Continue implementing controlfile and initfile support.
I've decided that the separator should be `/` instead of `_`.
This makes the translation to/from a flat control file to the hierarcal directory structure more straight forward.
Add the initial file load and wrap up most of the processing code.
The Rules still need to be processed.
The handling of the collapsed name needs to be implemented.
I've noticed that I don't have any string match functions in FLL.
I have string comparison functions, but these are not really match functions.
Or more precisely, the string comparison functions are exact matches for both strings.
Kevin Day [Thu, 20 Nov 2025 04:43:34 +0000 (22:43 -0600)]
Progress: Continue implementing controlfile and initfile support.
Change the code to pre-process and load all of the Rules during the Entry/Exit load.
The existing dynamic Rule file loading still happens, but only when not using a `controlfile`/`initfile`.
This, however, is unlikely to be used and that code might be gutted in the future.
Add new `F_load_not` status error for when the loading fails.
Update the error and debug printing functions to not use `debug` instead of `line_file`.
This brings the functions in line with the FLL debug printing functionality.
This still does not load the `controlfile`/`initfile` just yet.
Kevin Day [Wed, 19 Nov 2025 05:27:17 +0000 (23:27 -0600)]
Progress: Continue implementing controlfile and initfile support.
Fix issue where the `extern` prefix strings accidentally have `_length` in their names.
The entry and rule names should be without their prefixes.
Perform the prefix matching before pulling out the string.
Skip anything that doesn't match the correct prefix when using a loaded file.
Otherwise, operate normally.
This relocates the exit file loading to be at the start, right after loading the entry file.
It may be worth loading all of the rules specified in the entry and the exit at the start.
I added `@todo` and `@fixme` to represent these areas that I need to figure out how I want to handle.
Kevin Day [Mon, 17 Nov 2025 00:16:18 +0000 (18:16 -0600)]
Progress: Begin implementing controlfile and initfile support.
The specifications are now implemented.
Provide examples of the `controlfile` and the `initfile`.
Comments some TODO's that I am thinking about and will set aside for future consideration.
These TODO's will be removed once I wrap up the `controlfile` support regardless of whether or not I implement the TODOs.
The `controller` and `init` programs have the basic file support setup, but no function to actually process these are implemented yet.
My goal is to pre-process these files and then pass them along to the normal load functions.
Ideally, the file should need only be loaded in memory once.
I broke out some of the `controller_process()` functionality into another functions `controller_process_prepare()` and `controller_process_run()` for organizational purposes.
There are potentially some stale TODO/`@todo` comments that will be addressed and removed as I complete this functionality.
Kevin Day [Thu, 13 Nov 2025 03:05:26 +0000 (21:05 -0600)]
Bugfix: Program deadlocks on entry error.
The signal design by the standard libc signals and the stand pthread signals is rather lacking.
Sending a signal to the thread might cause it to deadlock because the signal handler is trying to exit but has locks and the cancellation function has called `f_thread_join()` on that signal handler thread.
First, switch to `f_thread_detach()` rather than join the thread.
This allows the signal thread to close gracefully without blocking the cancellation function.
A detached signal thread needs to know that once it is detached, it should not handle signals and should exit.
This is not straight-forward and pthreads fails to provide an easy way to do this.
(Where an easy way would be calling a function that gives the thread state of the current thread.)
The main signal ID could be checked for by getting the current thread ID and comparing the main signal ID to this current thread ID.
This works, but introduces bad threaded application practices.
The signal handler could read the main signal ID at any time before a potential change to it.
That is fine, it just missed the check in that case.
Still, this should be avoided as there could even be undefined behavior in this.
This takes an alternate approach of creating the `f_thread_attribute_t` and passing it to the thread.
The thread can then use this value to get the current detached state.
This should be much safer.
Due to the nature of threads detaching, this design requires that the `f_thread_attribute_t` to be de-allocated by the callback itself.
A new structure, called `controller_thread_arguments_t`, for passing this information to the thread is now provided.
The `f_thread_signal_write()` call handles certain libc functions but in other cases `f_thread_cancel()` should be called.
Add calls to `f_thread_cancel()` for any threads that have `f_thread_signal_write()` passed to it.
This helps ensure that both cancellation cases can be triggered.
These changes cause the interrupt behavior to be slightly different than previous operations.
I still plan on reviewing and determining how I want to make signal handling operate across the board for controller and init programs and for both entries and exits.
This behavior is therefore considered acceptable and provides a good chance to test out alternative approaches in practice rather than just in theory.
Kevin Day [Wed, 12 Nov 2025 05:52:57 +0000 (23:52 -0600)]
Update: Utilize the F_status_debug_source_d for handling the errors.
This simplifies some of the logic and code.
However, there are some disadvantages of doing this.
One such disadvantage is losing the specific function that has failed.
The location in the source file should be sufficient, if not ideal.
Address issues where the `debug` mode is not being specified in some programs.
Kevin Day [Sun, 9 Nov 2025 04:37:28 +0000 (22:37 -0600)]
Update: Support the status debug define.
The `controller_debug_file_line_d` has been implemented in FLL as `F_status_debug_source_d`.
Switch to using the FLL `F_status_debug_source_d` in place of `controller_debug_file_line_d`.
Kevin Day [Thu, 6 Nov 2025 06:26:29 +0000 (00:26 -0600)]
Update: Improve clean up function.
Add additional time check to reduce excessive instance clean up operations.
The reap condition gets triggered before the delay is resolved can result in constant clean up operations.
This increases the number of locks being grabbed just to check things.
A large number of processes can exit causes a large number of conditions to be triggered.
Avoid this by honoring the delay.
Only the reap gets performed if the delay is not yet expired.
Move the main instance clean up logic into a separate function `controller_thread_cleanup_perform()`.
Optimize the `while (F_false)` condition to not have a hard coded boolean in its condition.
Kevin Day [Thu, 6 Nov 2025 03:15:33 +0000 (21:15 -0600)]
Update: Wrap up changes to locking.
I am pretty much done with the locking changes.
I might want to do other changes pending additional review, but this is good enough for now.
Fix issue where a write lock is returning `F_lock_read` on error instead of `F_lock_write` on error.
Properly set write lock on child processes when assigning the `process_child_id`.
The `process_child_id` is a pointer to a memory region protected by locks and therefore needs to be write locked protected before changes are made to it.
Fix case where `controller_lock_read_instance()` is being called but its return value is not being checked for `F_interrupt`.
When `F_interrupt` is received in this (`sources/c/program/controller/main/rule/instance.c:152`), then break out of the loop and properly return interrupt status code.
The clean up thread should only try to lock the cancel mutex.
If a cancel is on going, then do not wait.
This should not wait because the clean up thread itself will be removed by the cancel operation.
Make a copy of the instances to operate and loop over the sets.
This reduces how long the instances lock shall be set.
The related instances lock can now also be a read lock rather than a write lock because no changes are to be made on the structure itself.
Ensure that the `controller_thread_instance_cancel()` function should exclusively wait for the lock rather than be interrupt driven.
This ensures that if the clean up thread is operating, then the instance cancel should wait for it to be finished.
Signal the reap condition before attempting to join the clean up thread to ensure that the reap lock is released, if it is held.
Kevin Day [Wed, 5 Nov 2025 02:49:05 +0000 (20:49 -0600)]
Progress: Continue changes to locking, improve debugging and error printing.
A recent work in progress commit resulted in the actions being incorrectly initialized.
This presents a good opportunity to improve the error reporting with the specific file and line number of the error.
This behavior can be disabled by turning off debug during build time.
The debug is enabled using the `debug` build setting mode.
This `debug` build settings mode is enabled by default.
Move the `controller_rule_instance_perform()` logic out and then collapse the `controller_rule_instance_perform_details()` functions into `er_rule_instance_perform()` functions to shorten their names.
This addresses some more locking issues resulting from many of the recent threading changes.
There is still a lot more work to do.
Particularly, there is addressing the problem causes the actions to be empty.
Kevin Day [Mon, 3 Nov 2025 04:03:04 +0000 (22:03 -0600)]
Progress: Continue changes to locking.
Change the rules array to be an array of pointers.
This better allows for locking and unlocking individual rules without having to hold locks on the rules array.
The rules array locks can now be shorter lived.
The stack itself needs to be an array of rule pointers now.
The index location is no longer needed or used.
This further allows for avouding needing locks on the rules array.
This refactoring is very much incomplete and controller probably does not work yet.
There are no locks available on individual rules yet.
This needs to happen!
I have added an explicit `@todo` in at least one location describing this.
Utilize the more granular error codes from the `f_thread_mutex_full_delete()` to handle the attibute delete even if the mutex delete itself fails.
I do not know how that should be handled.
There are still long term plans to make this program ever-running.
In such cases, all error states must be as recoverable as possible.
I doubt that I will get to this anytime soon.
Kevin Day [Wed, 29 Oct 2025 12:17:08 +0000 (07:17 -0500)]
Progress: Continue changes to locking, updating stand alone builds.
Synchronize with the FLL changes so that that stand alone builds again.
There were thread changes, including the removal of thread sets, that necessitate changing the config.h for stand alone builds.
Kevin Day [Wed, 29 Oct 2025 11:24:07 +0000 (06:24 -0500)]
Progress: Continue changes to locking.
Refactor `instance.lock` to `instance.use` to reduce the ambiguity regarding a lock named "lock".
Remove the `id` off of the instance.
This is not needed and removing it reduces the need for some locking.
The thread itself is now the "id" so `id_thread` is now renamed to just `thread`.
The `controller_instance_find()` function can now just get the instance pointer.
Add a "time" controller lock check flag.
Use this to designate expecting to return `F_time` on time outs.
Add additional documentation regarding locks.
Reduce uses of locking where possible by re-organizing code.
The `controller_instance_prepare()` function, in particular, has the the code changed to reduce how long the lock is held.
This is possible because the `controller_instance_find()` now gets the instance pointer rather than an identifier.
The `controller_instance_prepare()` function allocates the instance pointer independent of the array.
This allows for the write lock on instance to be only used when assigning to the array.
The instance must therefore be immediately freed on any error because it is not assigned to an array and cannot be auto-deallocated.
Add missing locks to areas where the locks should be, such as a read lock while reading the dependency rule status.
Make sure `instance.active` lock is set before calling `instance.use` locks (there may be more work needed regarding this).
Try to reduce how long `instance.use` are held (there may be more work needed regarding this).
Kevin Day [Sun, 21 Sep 2025 02:04:57 +0000 (21:04 -0500)]
Progress: Begin changes to locking.
I was investigating different locking change structures then I got interrupted.
I hadn't had a chance to get back to this yet and this commit is done to save the state.
These changes very likely will not work and I may end up doing this differently once I get back to this at a future date.
Kevin Day [Wed, 27 Aug 2025 02:39:33 +0000 (21:39 -0500)]
Update: Add specific source design documentation.
I have described this in one manner or another since the start of the FLL project.
This explicitly states the behavior that the FLL project (and related projects) have always followed.
Kevin Day [Fri, 15 Aug 2025 02:45:41 +0000 (21:45 -0500)]
Update: Additional lock tweaks.
I am thinking of doing some redesigning of the locking/unlocking processes after this commit.
I may end up adding a new type to the FLL project to standardize a more complex structure.
I believe I need to create a read/write lock pair with a mutex lock so that the mutex can be used to prep a lock for transitioning from a read to a write or a write to a read.
I need to do some more extensive planning.
Kevin Day [Thu, 14 Aug 2025 03:50:08 +0000 (22:50 -0500)]
Update: Add additional locking to the instance cancel.
Expand the locking functions to accept the seconds and nanoseconds.
Provide standard locking functions that do not need the seconds and nanoseconds passed.
Add locking with fail out status checks to the instance clean up function.
Kevin Day [Tue, 12 Aug 2025 02:51:13 +0000 (21:51 -0500)]
Update: Relocate the clean up thread thread cancel state.
Relocate the cancel state changes to outside both waits.
I do not like how this is now happening with a thread lock in place.
I will need to do further research and possibly further changes to better handle this.
For now, this is necessary so that when I send an interrupt signal that the clean up thread will cancel properly.
Kevin Day [Sun, 10 Aug 2025 02:22:48 +0000 (21:22 -0500)]
Bugfix: Do not use the main setting status with threads.
The signal thread and the other threads should not touch the main setting status.
Change the logic to ensure that the main setting status is left alone while threads are running.
Kevin Day [Sun, 10 Aug 2025 01:32:18 +0000 (20:32 -0500)]
Update: Major improvements in locking logic.
Major Improvements:
- Add custom thread status checking functions (also replacing `controller_thread_is_enabled()` with `controller_thread_enable_is()`).
- The thread enable property is now `enable` rather than `enabled`.
- Add new `controller_thread_enable_none_e` to allow for handling default behavior.
- Add new `enable` read/write lock to better handle the reading and writing of the enable property.
- Change the signalling logic to directly call `f_thread_condition_signal_all()` and to not read lock the conditions on signal send.
- Change the lock status checks to more explicitly check for `F_okay` so that unknown conditions without error bit set are ignored.
Being explicit on handling the `F_okay` helps ensures that the lock is guaranteed to be set.
I think I need to additional review and re-structuring of the thread logic beyond this commit.
There will likely be additional commits in the future to further improve the locking logic.
I believe that there are some cases where unlock might be called on a not-locked lock.
The additional work will likely include addressing this.
I suspect this is happening in cases where the same function is called in two different contexts.
One context is directly from a thread and another is in the middle of processing something.
The lock might not be set in the case of the directly calling from the thread.
Kevin Day [Sun, 3 Aug 2025 04:49:12 +0000 (23:49 -0500)]
Update: Thread locking and signal tweaks.
The `f_thread_condition_signal_all()` requires obtaining the mutex lock before operating.
Move this logic into a single function call and repeatedly use it to reduce code repetition.
The `controller_lock_signal()` is the function that handles the standard `f_thread_condition_signal_all()` call.
Rename `instance.wait' to `instance.wait_condition` to be more consistent.
Rename`instance.wait_lock` to `instance.wait` to be more consistent.
Add a `controller_lock_mutex()` that operates the same as `controller_lock_read()` but for a mutex lock.
The wait condition signal sender uses the newly added `controller_lock_mutex()`.
Use an inline static function `private_controller_rule_instance_perform_unlock_active()` to reduce some of the repeated `instance.active` unlocks in the function `controller_rule_instance_perform()`.
Change the cleanup thread to only unlock the `instance.lock` and `instance.action` locks when done.
I do have concerns with holding the lock before the `f_thread_join()`.
I will have to review and test this out some more.
Kevin Day [Sat, 2 Aug 2025 21:04:46 +0000 (16:04 -0500)]
Progress: Continue setup for granting init program the ability to handle shutdown/reboot.
Replace the `fll_program_standard_set_up()` call with a custom signal setup call (`controller_signal_set_up()`).
Both the init and controller programs might as well use the same signals.
The controller program should reap its own children too, even if it is not an init program.
Use a thread condition signal to communicate to the clean up thread to perform zombie reaping.
Add missing delete in the init program.
Make sure all condition signals have proper checking around the success/failure of the condition listener.
Make sure all condition broadcasts lock before sending the signal and unlocks after.
This too must be checked for success.
Send a reap condition signal to wake up the clean up thread before cancelling the instance.
Add custom thread names using recently added thread naming FLL functions.
This should make analysis and observation of threads at runtime far easier.
The clean up thread is now located sooner in the process so that the zombie reaping may be processed during entry operations.
This should allow for the init/controller programs to receive the signals.
The init program is not yet written to do anything with them yet.
I tested the serial test like this:
```
controller -s data/data/controller/example/miscellaneous/ serial
```
I noticed that the interrupt signal before the wait is reached causes the program to exit.
Once the wait is reached and an interrupt is then sent then the exit process is started before exiting.
I remember being uncertain how to handle this and left that logic to future decisions.
As of now I believe that the exit process should always be called on interrupt.
This should be addressed ideally along with this shutdown/reboot handle or at the very least immediately after that.
Kevin Day [Thu, 31 Jul 2025 02:51:02 +0000 (21:51 -0500)]
Progress: Initial setup for granting init program the ability to handle shutdown/reboot.
The standard shutdown/reboot/ctrl-alt-delete process actually operates via interrupts.
Make the init program interruptible by default and add notes in the help.
Add additional callbacks.
The help callback is added to print additional information specific to the init program.
The process thread signal is going to be a new callback that the init program can use to define custom behavior.
This custom behavior will utilize the special shutdown/reboot/ctrl-alt-delete signals.
The init program will be handling more signals than the controller program.
(I may want to consider looking into supporting more signals for the controller program as well.)
Kevin Day [Wed, 30 Jul 2025 02:42:38 +0000 (21:42 -0500)]
Bugfix: Signal thread is not cancelling when f_thread_cancel() is used.
Change the behavior to directly send the terminate signal using `f_thread_signal_write()`.
This appears to get the signal handler thread to terminate when needed.
The simulate validate mode now operates without hanging and without required a terminate signal.
Set the thread cancel state to `PTHREAD_CANCEL_ASYNCHRONOUS` while waiting for a signal.
Set the thread cancel state back to `PTHREAD_CANCEL_DEFERRED,` while not waiting for a signal.
Kevin Day [Tue, 29 Jul 2025 02:57:38 +0000 (21:57 -0500)]
Bugfix: Instance thread can deadlock on interrupt.
The instance thread can deadlock on interrupt because the thread enable is not being checked before performing the lock.
Add a wait and retry lock that includes checking if enabled.
Kevin Day [Tue, 29 Jul 2025 02:30:41 +0000 (21:30 -0500)]
Bugfix: Final clean up thread deadlocks on interrupt.
There is a sleep at the top of the while loop in the clean up thread.
This sleep does not receive interrupts when `PTHREAD_CANCEL_DEFERRED` is set as the cancel state on the thread.
Set the cancel state to `PTHREAD_CANCEL_ASYNCHRONOUS` during the execution of the loop.
This is safe because there are no special locks being held during this sleep at the top of the loop.
Restore the cancel state to `PTHREAD_CANCEL_DEFERRED` once past the sleep.
Add additional checks in the loops for `controller_thread_enabled_e` to potentially catch any situations where an immediate exit would be ideal.
Make sure all locks are unlocked before breaking out of any part of any loop.
Use a try lock when sending signals to child processes rather than a full lock to increase the chance of handling any interrupts.
Kevin Day [Sun, 6 Jul 2025 21:08:28 +0000 (16:08 -0500)]
Update: Next micro version (0.7.3).
The following are the commands that I ran to make this change:
# find data/ sources/ specifications/ documents/ licenses/ install.sh -type f -exec sed -i -e 's|0\.7\.2|0.7.3|g' -e 's|^version_micro 2|version_micro 3|g' '{}' ';'
# find sources/ -name *.h -exec sed -i -e 's|_program_version_micro_s F_string_ascii_2_s|_program_version_micro_s F_string_ascii_3_s|g' -e 's|_program_version_micro_s_length F_string_ascii_2_s_length|_program_version_micro_s_length F_string_ascii_3_s_length|g' '{}' ';'
# sed -i -e 's|version_micro 2|version_micro 3|g' data/build/settings* data/build/stand_alone/settings.*
Kevin Day [Sat, 28 Jun 2025 03:40:57 +0000 (22:40 -0500)]
Cleanup: Apply new script practices.
Be more consistent about wrapping all variables in brackets.
This helps make the code more consistent and also more compatible with other interpreters like ZSH.
Don't use echo pipe syntax and instead use "<<<" redirection pipe syntax for passing variables to program calls.
The grep commands should use either `-sho` or `-shoP`.
Kevin Day [Fri, 6 Jun 2025 02:42:16 +0000 (21:42 -0500)]
Security: Explicitly define IFS to prevent misuse.
The scripts are written with certain expectations.
This expectation may not be properly met if the `IFS` value is changed.
This can potentially be used to create some sort of exploit.
Explicitly define IFS and then do so at a local variable scope to prevent affecting the callers IFS setting.
Kevin Day [Fri, 9 May 2025 02:47:39 +0000 (21:47 -0500)]
Bugfix: Relocate static libcap linkage to after the libfll linkage.
The parameter order apparently matters when it comes to linking static libraries.
Perhaps the `-l:libXXX.a` format (instead of `-lXXX`) is where this order somehow matters.
Kevin Day [Thu, 8 May 2025 04:52:33 +0000 (23:52 -0500)]
Workaround: The compiler and linker fail to understand that compiling statically should also link statically.
This is a logic or design flaw that is just plain idiotic.
When compiling statically (passing `-static`), there cannot be any shared/dynamic linked data.
Rather than recognizing this, if there is a shared library during a static build then the shared library is linked to.
The compiler and linker, at least, supports explicitly forcing the point that a static library is to be linked.
This is done via `-l:libc.a` for libc linking rather than `-lc`.
This should not be necessary, but it unfortunately is.
Kevin Day [Thu, 1 May 2025 01:46:56 +0000 (20:46 -0500)]
Update: Next micro version (0.7.2).
The following are the commands that I ran to make this change.
# find data/ sources/ specifications/ documents/ licenses/ install.sh -type f -exec sed -i -e 's|0\.7\.1|0.7.2|g' '{}' ';'
# find data/ sources/ specifications/ documents/ licenses/ -type f -exec sed -i -e 's|^version_micro 1|version_micro 2|g' '{}' ';'
# find sources/ -name *.h -exec sed -i -e 's|_program_version_micro_s F_string_ascii_1_s|_program_version_micro_s F_string_ascii_2_s|g' '{}' ';'
# find sources/ -name *.h -exec sed -i -e 's|_program_version_micro_s_length F_string_ascii_1_s_length|_program_version_micro_s_length F_string_ascii_2_s_length|g' '{}' ';'
# sed -i -e 's|version_micro 1|version_micro 2|g' data/build/settings* data/build/stand_alone/settings.*
Kevin Day [Tue, 29 Apr 2025 22:48:39 +0000 (17:48 -0500)]
Update: Synchronize with FLL release by setting version to 0.7.1.
In the longer term the version numbers will likely be on a different number.
In the short term, as in now, it is much easier to just maintain them at the same release versions.
Kevin Day [Tue, 28 Jan 2025 04:45:43 +0000 (22:45 -0600)]
Update: Improve signal handling code.
Move the `fll_program_print_signal_received()` calls to the main program to help ensure consistent printing at the end of the program.
- Note: I am not certain if I want this instead in the main() function or not and I opted not to do so for now.
Kevin Day [Wed, 11 Sep 2024 22:32:38 +0000 (17:32 -0500)]
Security: Changes to f_memory_array_increase() API causes invalid memory access.
This is a security related regression.
The `f_memory_array_increase()` has been changed to only guarantee that at least 1 element is increased if not available.
The code is Controller is depending on the old behavior where the allocation step is guaranteed.
The new behavior of `f_memory_array_increase()` performs additional steps to prevent memory abuse which in tern causes the first allocation to only consist of a single element.
Use instead `f_memory_array_resize()` if the size is too small.
In another case, instead use `f_memory_array_increase_by()` to ensure that the increase is at least 2 elements.
Kevin Day [Wed, 31 Jul 2024 02:45:38 +0000 (21:45 -0500)]
Refactor: Change bit-wise enumerations into defines.
I did some reviewing of how the enumerations used for flags are used.
These generally are not being used as a type.
An enumeration slightly increases the resulting binary size.
Enumeration values might be limited to just type of int.
This seems like an easy (small) optimization to just use defines rather than enumerations for flags and other bit-wise numbers.