How to Configure Multithreaded C32 Serial Comms: Tutorial

This page describes how a device manufacturer can configure the grouping of CSYs into various threads.

The Serial Server on Symbian platform supports the use of multiple threads in order to separate the serial plug-in CSY modules. The first section provides a brief migration guide, while the subsequent sections provide more detail on the specific parameters and behaviour, and other advanced information.

Introduction

The Comms Configurator (c32start) uses a configuration Comms Module Ini file (.CMI) to load Comms Provider Modules (CPMs) into the Rootserver process. Each CMI file represents exactly one CPM loaded inside Rootserver. CPMs are the name given to threads running inside the Rootserver process, since they extend the thread concept by adding a wrapper of extra information and functionality used to manage the threads.

The Comms Configurator searches for the CMI files in the \private\101f7988 folders by searching from Y: to A: and then Z:. The folder 101f7988 is the secure area for the Comms Configurator process and its name derives from the UID for the process - as specified by the requirements for Data Caging. If the files are found in the ROM (usually Z:) they are not then copied to any other drive but only read in-place. Thus, a later configuration of CMI files installed on another media/drive other than the ROM drive can be used to override the original ROM configuration. The Configurator reads these files only once during boot so any further changes will not take effect until the next system boot.

Quick Migration Guide

This section provides a brief description of how to migrate a configuration from the old single-threaded C32 Serial Server to the new multi-threaded server. As there are numerous potential configurations with the multi-threaded server, this migration guide should be used as a first step to getting any old settings migrated over and thus allowing the server to continue working after an upgrade of Symbian platform.

Once migrated, the other sections in this article provide further useful information regarding configuring CMI files for better performance in line with the requirements of the particular device.

The quick migration involves two steps:

apply any special configuration used with the previous single-threaded C32 Serial Server to the new configuration files supplied with the multi-threaded server
adjust the default set of configuration files supplied with the multi-threaded server to suit the particular serial plug-ins in use

These two steps are explained in more detail in the following sections.

Migrating settings

The original single-threaded Serial Server was controlled by a single configuration file called (by default) c32.cmi, located in \private\101f7988. A default version of this file shipped with Symbian platform, although a device manufacturer could change this file. The default version of this file was:

## Comms Configurator config file for the C32 server

[Loader]
Name=CCommServer
FileName=c32.dll
ThreadFunctionOrdinal=87
IsServer=1
IsSticky=1
Priority=EPriorityMuchMore
StackSize=8192
ScaledStartupState=0x1085
HeapOption=EDefaultHeap
MinHeapSize=256
MaxHeapSize=524288
SystemCritical=1

The multi-threaded C32 Serial Server (C32MT) ships by default with five configuration files, since by default it supports five different threads. These configuration files are:

\private\101f7988\c32.cmi
\private\101f7988\c32_first.cmi
\private\101f7988\c32_second.cmi
\private\101f7988\c32_third.cmi
\private\101f7988\c32_fourth.cmi

They are an extended version of the original CMI file that shipped with the single-threaded C32. For example, the first of these configuration files (c32.cmi) looks like:

# c32_main - Comms Configurator config file forming part of the Default C32 server configuration

[Loader]
Name=C32_Main
FileName=c32.dll
ThreadFunctionOrdinal=87
IsSticky=0
IsServer=1
Priority=EPriorityMuchMore
StackSize=16384
ScaledStartupState=0x1085
HeapOption=ENewHeap
MinHeapSize=256
MaxHeapSize=524288
SystemCritical=1
Group=C32SerComms
OnDemand=0
# Don't check heap if shutting down with EImmediate (KCF_ModuleImmediateNoCheck)
ControlFlags=4
[IniData]
Role=Dealer
WorkerId=0

The first step in migrating is to ensure that any unique configuration used with the old server is migrated across to the new server. The particular parameters that should be considered in this procedure are:

Priority
StackSize
ScaledStartupState
MinHeapSize
MaxHeapSize
SystemCritical

These parameters would need migrating to each of the five new CMI files. The following notes relate to deliberate changes to parameter values between the single and multi-threaded servers:

HeapOption: The value of HeapOption is different between the old and new files - this was made to aid in debugging the new multi-threaded server by separating its memory from the rest of the comms system. It can be safely changed back if desired.
IsSticky: The value of IsSticky has changed from 1 to 0. The IsSticky parameter is normally unused in a production system for the Serial Server since the server runs as a SystemCritical thread and thus cannot be shutdown and, if it panics, the phone would reboot. The change was made to facilitate testing, since in a test environment the Serial Server can be shutdown, and to facilitate this the value of IsSticky must be set to 0.
StackSize: The value of StackSize has increased from 8192 for the single-threaded C32 to 16384 for C32MT. This change is due to the increased stack usage of the multi-threaded C32. The value used - 16384 - was based on the value proven to work with the similar ESock multi-threaded server. It is possible that this value could be trimmed back towards 8192 to save memory depending on the CSYs being used, since the ECUART CSY has been tested to work with the multi-threaded C32 server with a stack size of just 8192, and CSYs are usually much less memory-demanding than the protocols being used in ESock.

Assigning CSY plug-ins to threads

The second part of migration to the new multi-threaded serial server is to alter the default set of CMI files to match the plug-ins (CSYs) in use and the desired operation. This step is not mandatory since by default C32 will load all plug-ins correctly. But, it is worth considering at this stage whether the default configuration needs some changes to match the requirements of the device.

There are three broad configurations available, which are defined by whether one or more CSYs are run in the main thread for efficiency. The default configuration supplied with Symbian platform runs all the CSYs outside the main thread since this ensures that no CSY can deny service to the main server thread. It is expected that this configuration will suit most situations, but for more information on the other configurations, see the later section Typical Thread Configuration Examples.

As mentioned earlier, the default configuration uses five threads. The description for each of these is given in the following table:

Worker ID	Summary	CMI file	Description
0	Main thread	`c32.cmi`	Runs the serial server to service client requests and pass on to the CSYs, possibly in other threads. This thread is expected to be started at device boot.
1	USB thread	`c32_first.cmi`	Runs the USB CSY (ECACM). This is separated from all other threads so that USB can run at high-speed uninterrupted. This thread is only started when a client requires a USB connection.
2	Baseband thread	`c32_second.cmi`	Runs the RS232 CSY (ECUART). This thread runs the ECUART plug-in separately since ECUART is sometimes used to connect to the baseband. If a different CSY is being used to connect to the baseband, it can be added to this CMI file and ECUART either moved to another thread, or removed altogether. As the baseband requires a high speed and fast response time, it is recommended that the baseband CSY should be run in this thread to separate it from all other CSYs. This thread is only started when a client requires the CSY.
3	Bluetooth and IrDA thread	`c32_third.cmi`	Runs the Bluetooth and IrDA CSYs. These CSYs usually do not have high requirements for throughput or response and thus can run together. This thread is only started when one of its CSYs is required by a client.
4	Default thread	`c32_fourth.cmi`	Runs any unlisted CSYs. If C32 is asked to load a CSY for which no thread has been identified in a CMI file, then C32 will load the CSY in this thread. This thread is only started when required.

For a particular device, the above configuration may need to be changed to address the requirements of the device. Typical adjustments to consider are:

Change the CSY name for Worker ID 2 to match the baseband CSY, and then either move ECUART to Worker ID 3 or into its own new thread, or don't add it to any CMI file if the device doesn't use the ECUART CSY
Add any extra low-performance CSYs to Worker ID 3 so that they run together with Bluetooth and IrDA. Extra CSYs could be run in a new separate thread but there is a memory cost for each new thread added so this should be considered (8K is typically allocated to a new thread, see Process, thread, stack and memory useful information, and there is a limit to the number of threads with their own heaps that can be started in the Root Server).
For any extra high-performance CSYs, create a new CMI file so that these can run separately from the other CSYs. See Creating A New CMI below
Prioritise important threads (or reduce the priority of lesser threads). Candidates for increased priority would be the Main, USB and Baseband threads. Care should be taken when adjusting thread priorities since these affect the whole Comms system - more guidance on setting thread priorities is available in the document How To Assign Thread Priorities elsewhere in this developer library.

When making these adjustments note that there is no cost involved with mentioning CSYs that may never be loaded.

Overview of the C32 multi-threaded CMI files

C32 Multithreading uses the same Dealer/Player terminology/concepts as Esock Multithreading. The Dealer is the main C32 Comms Provider Module (CPM) which runs the C32 server; a Player is a C32 CPM that only runs one or more CSYs and takes the client requests from the Dealer module.

For example:

C32.cmi

[Loader]
Name=C32_Main
FileName=c32.dll
ThreadFunctionOrdinal=87
IsServer=1
IsSticky=0
ThreadPriority=EPriorityMuchMore
StackSize=16384
ScaledStartupState=0x1085
HeapOption=ENewHeap
MinHeapSize=256
MaxHeapSize=524288
SystemCritical=1
Group=C32SerComms
OnDemand=0
[IniData]
Role=Dealer
WorkerId=0

The ThreadFunctionOrdinal identifier must refer to the correct ordinal number of the main thread function for the DLL file containing the Comms Provider Module. Due to binary compatibility considerations this must be placed at ordinal "87" in the export table. A Comms Provider Module/CPM is essentially a thread managed by the Comms Configurator.

The description of each of the settings under [Loader] that are specific to C32 and the settings under [IniData] are described in further detail below.

As well as the C32 server thread, any other CPMs that C32 needs must also have their own CMI file. One such example file is:

C32_Default.cmi

[Loader]
Name=C32_Default
FileName=c32.dll
ThreadFunctionOrdinal=87
IsServer=1
IsSticky=0
ThreadPriority=EPriorityMuchMore
StackSize=16384
ScaledStartupState=0x3040
HeapOption=EShareHeap
SharedHeapName=C32_Main
MinHeapSize=256
MaxHeapSize=524288
SystemCritical=0
Group=C32SerComms
OnDemand=1
Binding0=C32_Default:4,C32_Main:0,ECustom,42,42
[IniData]
Role=Player
WorkerId=4
CSYList=*

C32 CMI Parameter Descriptions

This section gives details about the parameters available in the CMI file relevant to C32. The CMI file has two sections - a [Loader] section and an [IniData] section. The [Loader] section parameters are defined for the CPM, and the ones relevant to C32MT are detailed below. The [IniData] section contains parameters specific to a CPM. If further information on the format of the CMI file itself or the [Loader] parameters is required, please refer to the document Comms Process Starter/Config Design Document section 3.1.1. This document is located in this developer library as well as in the comms-infras\rootserver\Documentation folder.

A note on CMI processing

No mechanism is provided to change CMI files at run-time, since they are designed to be configured for inclusion in a ROM and then not changed through the life of a phone product. After-market changing of the configuration is supported via the installation of CMI files in a non-ROM folder which then take precedence over the ROM files when the device is next rebooted. Thus, any CMI configuration file problems should become apparent during the early stages of phone development.

A bad configuration will panic in debug builds to signal the user to change/edit the .CMI files in order to make a desired and correct configuration. In urel builds a bad configuration may survive (unless the .CMI files are very malformed) but at a cost that all CSYs are loaded in the Dealer Thread (Workerid=0) making it in effect a single threaded system.

The panic numbers used by the C32 Serial Server for internal server faults are defined in the enum TECommFault.

CMI [Loader] Parameters relevant to C32

OnDemand

This indicates whether the CPM is started as part of the Configurator’s CPM load on boot ("0"), or whether the CPM should only be loaded via an explicit request to Configurator ("1"). The default value is "0". The OnDemand parameter for the main thread can only be "0" if mentioned. For Player modules the setting can be either "0" or "1". If a Player thread is not marked as OnDemand=1 then it will load during the boot sequence. If the main thread (Workerid=0) is marked as OnDemand=1, then this is a configuration error and C32 can never be started (although in theory this is possible - see Footnote 1 below). Players marked to load at boot up do not load any CSYs implicitly as part of the boot load.

All Player modules for C32 must be marked with OnDemand=1 since the C32 Dealer must instigate a Player load. If a Player thread is not marked as OnDemand=0 then it will load during the boot sequence.

Group

All CMI files for C32 must map to the same Group ID. The actual Group ID for the C32 group is C32SerComms. CMI files for C32 Players that do not have a Group member will never be loaded by C32 (assuming OnDemand=1 is set for the Player threads).

If there is no group defined in any CMI files and therefore no [IniData] is processed for any of them, then C32 will simply assign all CSYs to load into the Dealer thread.

Name

This parameter is used to identify the CPM to the Configurator and is used to name the thread for the kernel. It is stored by C32 for all CMI files in the C32SerComms group for use when requesting Configurator to load or unload a CPM. As this value is easily configurable and is used to name the thread to the kernel, software should not rely on the name of any C32 thread being any particular string. Prior to the multithreaded C32 (C32MT), the C32 thread was conventionally known as CCommServer. However, with C32 now potentially having a multiplicity of threads, other schemes are probable such as C32_Main, C32_USB and C32_BaseBand or C32_BT-IR and C32_Default

The Dealer and all Players in the C32 group can have any name since C32 will find them as part of the same group (see Group above). The names in the C32 group should all be unique – if the same name is found twice or more C32 will panic in debug builds but attempt to recover in release builds by only processing the first that it finds. This is still a highly risky situation since if C32 needs to start that module in order to accommodate a CSY, Configurator may load the wrong CMI file when creating the new thread.

When introducing new CMIs, care should be taken to change the BindingN line in CMI to reflect the new modulename and WorkerId.

CMI [IniData] Parameters relevant to C32

In the case where C32 has a CMI file that loads the C32 binary during boot up, but the CMI file does not have an [IniData] section, C32 will assume that this CMI is for the main thread and that the CMI is a pre-C32MT file. However, for C32MT, this is an undesired configuration and should be rectified. Old versions of c32.cmi are only kept in the source distribution to support backwards compatibility and should not be used in a system which has C32MT present. In such backwards-compatible modes of C32MT, the main thread will host a Player in case no other CMI files are present.

Role

The value for this parameter should be either Dealer or Player. Only one Dealer can exist in a C32 system, but there can be any number of Player s. If there are two or more Dealer CMI files, the ScaledStartupState determines which Dealer CMI starts first. If both the CMIs have the same ScaledStartupState, then they are loaded in alphabetical order. Configurator is guaranteed to start a C: CMI file before the Z: one. If they both have the same module name then it is safe since Rootserver will refuse the load of the second module with the same name failing with KErrAlreadyExists. However, if the module name is different Rootserver will load them both and C32 will detect that an instance of itself is already running and exit the second thread. Only the IniData for the CMI file that was used to start the Dealer will be processed, with all other Dealer CMI files being ignored. If the main thread is to include a co-resident Player, the iniData should include the CSYList key-value pair discussed below.

WorkerId

The ID number of this thread inside the C32 system. The Id numbers should be a positive integer or "0" in the case of the main thread (In case CMI file of WorkerId=0 has anything else than Dealer as role, it is overridden with Role=Dealer, logging a warning). The system expects subsequent ID numbers to be incremented up from the main thread's ID. Where a CMI file in the C32 group does not have a WorkerId, it is a bad configuration and will panic in debug builds. Allocating the next higher number is not possible as this Player needs to be bound with the Dealer using the BindingN line which accepts WorkerId parameter. In case of duplicate WorkerId, a panic occurs as the second module cannot be bound to the Dealer as the binding for the main thread has already been done.

CSYList

A comma-separated list of the filenames of the CSYs this thread will own. All CMIs in the C32 group must have a CSY list, except for the Dealer which can optionally have a CSYList. A Player with no CSYList represents a bad configuration as a Player’s sole purpose is to load CSYs. Such a Player with no CSY list will never be started by C32. A Dealer with no CSY list will act purely as a Dealer. A Dealer with a CSY list becomes a Dealer with a co-resident Player inside its thread. CSYList param determines if the CPM will host a Player or not.

If a CSY appears in more than one CMI file, then the CSY will be assigned to one thread, but not necessarily that with the least number of CSYs in it.

The comma-separated list should not have any spaces or other white space in it, and each CSY name must be an alphanumeric string. The CSY name is not case sensitive, and should not have the .CSY extension. C32 does not check whether the supplied CSY names actually exist as binaries, since it only refers to the list when a client asks to load a particular CSY.

If an asterix ("*") appears as a CSY name in the CSY list, this is inferred by C32 as meaning that this thread is also to be used to load any CSYs not mentioned in any CMI file, and is known as the default thread. If two different CMI files in the C32 group have the wildcard then only one CMI file will be marked as being the host for unknown CSYs – it is not defined as to which one. If no CMI file has a wildcard then C32 will assign the wildcard to an unused Player thread so that all unlisted CSYs are isolated into their own thread. This is overridden if the only CMI file is for the Dealer, in which case the Dealer will load all CSYs into its own co-resident Player, since the Dealer cannot request the Configurator to start a module for which there is not an associated CMI file.

Important Notes

Field names in the [IniData] are case sensitive, but fields values are not.

The BindingN line is responsible for binding the Player CPM with Dealer. If you leave out the BindingN line from the CMI then the new Player will in essence remain as an island and no requests for loading or any subsequent operation can be passed to it from Dealer. This will not result in a panic.

The C32_Main.cmi doesn’t have a BindingN line since it is a Dealer and thus does not need to bind to another Dealer.

Creating A New CMI

To create a new CMI file, copy an existing Player CMI file, rename it and change the necessary parameters as detailed below. The examples below assume that the base CMI used was the supplied c32_fourth.cmi. The following parameters need to be changed to make a valid configuration file:

In [Loader] section are:

Name - This is the module name of CPM.
BindingN - BindingN line is changed to reflect the new module name and WorkerId value

In [IniData] section:

WorkerId - This value is numeric and must be unique
CSYList - Contains CSYs which you want to load in this new CPM, separated by commas, no white spaces allowed.

Since the additional CMI being added is a Player, the Role field remains the same and it can not be changed to Dealer as the CPM with WorkerId=0 is the one and only Dealer in the C32MT.

[Loader]
# Change the Name value to your CSY name
Name=C32_Default  //

Change the Name value to the name of your CSY, e.g.

Name=C32_MyCSYs

FileName=c32.dll
ThreadFunctionOrdinal=87
ThreadPriority=EPriorityMuchMore
StackSize=16384
ScaledStartupState=0x3040
HeapOption=ENewHeap
MinHeapSize=256
MaxHeapSize=524288
SystemCritical=1
Group=C32SerComms
OnDemand=1

Change the BindingN line to refer to your CSY, for example change from

# Change the BindingN line to refer to your CSYs
Binding0=C32_Default:4,C32_Main:0,ECustom,42,42

Binding0=C32_MyCSYs:5,C32_Main:0,ECustom,42,42

And in the [IniData] section:

[IniData]
# Make sure Role is set to "Player"
Role=Player

Change the WorkerId, for example from WorkerId=4 to

WorkerId=5

and change the CSYList to specify your CSYs, e.g. change "CSYList=XX,YY,zz" to

CSYList=MyCSY1,MyCSY2

Typical Thread Configuration Examples

This section describes the three broad configurations for threading available for the C32 Serial Server. Each has different potential effects on the performance of C32.

The three broad configurations available, with a summary of their costs and benefits, are detailed in the following table:

	Configuration	Advantages	Disadvantages
1	Run all CSYs outside the main thread [Default supplied CMI set]	Some CSYs can be promoted ahead of others using thread priorities Main thread cannot ever stall when any CSY stalls Stackable CSYs are possible: a CSY can connect to another CSY through the C32 API as long as both are in different threads	Operations on CSYs are a bit slower due to increased overhead of inter-thread communication Complex to debug
2	Run some CSYs in the main thread	CSYs in the main thread avoid the overhead of inter-thread communication Some CSYs can be promoted ahead of others using thread priorities Main thread cannot stall when a CSY in a separate thread stalls Stackable CSYs are possible: a CSY can connect to another CSY through the C32 API as long as both are in different threads	If any CSY in the main thread stalls, the whole server stalls No means to prioritise one CSY ahead of others (without changing the CSY code) Operations on CSYs outside the main thread are a bit slower due to the increased overhead of inter-thread communication Complex to debug
3	Run all CSYs in the main thread [Default configuration when any CMI files are corrupt]	Simplest to debug since no inter-thread communication All CSYs avoid the overhead of inter-thread communication	If any CSY stalls, the whole server stalls No means to prioritise one CSY ahead of others (without changing the CSY code) No stackable CSYs are possible

The default set of CMI files supplied with Symbian platform is based on the first broad configuration since it was decided that the small cost of running all CSYs outside the main thread is justified by the increased reliability to the whole C32 system of ensuring no CSY can deny service to the C32 server. Each of these configurations is explained in more detail below.

Figure: Figure 1 - No CSYs in main thread: RootServer with 4 C32 CPMs

Referring to Figure 1, the following can be observed:

The Main Thread has WorkerId=0, Role as Dealer and is configured to load no CSY
The Player Thread 1 has WorkerId=1, Role as Player and is configured to load ECACM CSY
The Player Thread 2 has WorkerId=2, Role as Player and is configured to load ECUART CSY
The Player Thread 3 has WorkerId=3, Role as Player and is configured to load unlisted csy. What unlisted csy means here is that apart from ECACM and ECUART, if any new csy is to be loaded, it will get loaded in this thread.

The [IniData] section for the above CPMs would look like:

[IniData]
Role=Dealer
WorkerId=0

[IniData]
Role=Player
WorkerId=1
CSYList=ECACM

[IniData]
Role=Player
WorkerId=2
CSYList=ECUART

[IniData]
Role=Player
WorkerId=3
CSYList=*

Figure: Figure 2 - All CSYs in main thread: RootServer with single C32 CPM

In Figure 2 the CPM loads any CSYs, and so should contain CSYList tag in the IniData section.

The Main Thread has WorkerId=0, Role as Dealer and is configured to load any CSY. In this configuration since it is a single thread, all the CSYs will be loaded in the same thread.

The [IniData] section for above CPM would look like:

[IniData]
Role=Dealer
WorkerId=0
CSYList=*

Figure: Figure 3 - Some CSYs in main thread: RootServer with 3 CPMs

From Figure 3, the following can be observed.

The Main Thread has WorkerId=0, Role as Dealer and is configured to load HSDPA csy
The Player Thread 1 has WorkerId=1, Role as Player and is configured to load ECACM csy
The Player Thread 2 has WorkerId=2, Role as Player and is configured to load an unlisted csy

The IniData section for above CPMs would look like:

[IniData]
Role=Dealer
WorkerId=0
CSYList=HSDPA

[IniData]
Role=Player
WorkerId=1
CSYList=ECACM

[IniData]
Role=Player
WorkerId=2
CSYList=*

Footnotes

Footnote 1: In theory the C32 main thread (Workerid=0) could be marked as on-demand and started by the first client to connect to it. This might be useful since it would save device boot time for a device that does not need serial comms as part of boot up. The process would require the client to connect to Configurator and load the C32 main module explicitly. For this to work the main Module’s name would have to be fixed (the reference system uses “C32_Main ”) so that the client can identify it to Configurator - which is not a big problem, and the client would need the appropriate capabilities (also not a big problem as the client could well be inside Rootserver too). The other problem is that making it on-demand requires that it is still started by a priviledged process before any unpriviledged process attempts to use the C32 API (since this will fail unless no CSYs require any capabilities). Therefore, for on-demand main thread-loading to work, intimate details about the specific configuration of CSYs for a device and their potential clients would be needed.