View all our documentation at www.CarbonORM.dev!

The CarbonORMs are sometimes called C6 since Carbon is the sixth element in the periodic table of the elements. We aim to enable developers to query SQL in their middleware and frontend in a safe, constant, and syntactically pleasing way. Our Rest-based ORM auto-generates 1-1 type references to your tables and columns in JavaScript, TypeScript, and your middleware language (PHP, JAVA, PYTHON, C#, ...). We are still working to support all major languages, and help is very much welcome! Payloads between the frontend and middleware are JSON based, and working with SQL in your middleware will use objects and array language-specific datatypes. Our generated C6 typings ensure a few things:

1. Large organizations with a frontend and backend team separated will not see the timing issues typical projects face. Generally, most operations can be done using a single table, and all connections are already generated to interface from the front end; front-end teams will not have to wait for the backend to create the necessary routes required for new features. This leaves backend teams focusing on validations rather than boilerplate and the frontend without bottlenecks.
2. Type definitions allow your editors to make better coding suggestions.
3. If you change a column name, every internal query in your project middleware that uses the column constant can be found and changed. Depending on the middleware, accessing the old value would cause a compile-time or invalid reference error. This reduces the load on your SQL servers and in many cases will prevent publishing invaild code all togeather.
4. Invalid external requests, think bad-actor, that use invalid column references raise an Error in the middleware, not the backend SQL server reducing load and possible Distributed Denial-of-Service (DDOS) abuse potentially affecting every user.
5. Teams of developers can ensure that the SQL data schema stays up-to-date with each other. We have identified general change rules that allow CarbonORM to update projects on different servers or databases automagically. Some examples include auto-creating columns, primary and foreign key references, and changing column types. CarbonORM has also standardized where making advanced table updates should be placed. When changes exist that cannot be deterministically automated by us, an error will be thrown!

This concept began as a simple PHP project and have slowly began integrating with NodeJs and the React Frontend runtime. Other language repositories are in progress but no other are ready to be use in a production setting. Focus for future development has been restricted to minor and patch releases in CarbonPHP. CarbonNode will gain many of the feature CarbonPHP already offer. Development in CarbonC will begin and take precidence to help introduce module support for many laguages with one package. For example Ruby, Python, Node, and PHP are all written in C so writing CarbonORM in C is an enevitabillity that will increase preformance for all of these higher level languages

Most languages support prepared statements, but the boilerplate code becomes repetitive in large projects. We eliminate the need for repetition by looping through these data structures and validating against our auto-generated code data definitions.

In local environments, we don't ;) This allows front-end engineers to move quickly. In our CI/CD pipelines, the frontend tests will quietly store all prepared statements it queried during the test. Our ORM conveniently provides this information with every request. After every test completes, C6 will compile these 'Valid SQL procedures' into a single object that the middleware will use for its validation in non-local environments. Any SQL executed from the frontend not existing in our valid SQL array will be logged and rejected.

While editing generated code is typically insufficient, we look to change this common practice. We believe there is no better place to put restful helper methods than the table class they operate on. We use custom complex regexes to capture these custom methods and persist them from revision to revision. We are pleased about how seamlessly this works and improves the DevOps experience.

It's possible to process complex Transactional code, kill a request based on the SQL generated, validate a column and its arguments each time it is processed, preprocess the request before the rest code is run, run global validations on every column encountered, rollback a transaction based on the results of a query, and even more possibilities.

Queries can be locked using MySQL InnoDB ^8 FOR UPDATE and FOR SHARE, which we can be used by adding self::LOCK => self::FOR_UPDATE, and self::LOCK => self::FOR_SHARE to the query body.

As C6-enabled SQL queries are just objects and arrays, queries can be stored and abstracted in variables or constants and modified without string manipulation. Hooray! Middleware languages have varying syntax for representing these datatypes. To view language-specific examples for middleware queries, please check out your respective middleware repository:

• All Organization Repositories
• CarbonPHP
• CarbonNode
• CarbonReact
• CarbonJava
• CarbonPython

We will use the standard frontend JavaScript (TypeScript) syntax for our examples below. GET requests are JSON based and serialized into query parameters using qs.

Qs.stringify(params, {arrayFormat: 'indices', indices: true, skipNulls: false, strictNullHandling: true})

QS is a dependency of CarbonNode and thereby CarbonReact. It will automatically be added to your package when you install either.

"@types/qs": "^6.9.8",
"qs": "^6.11.1",

httpd and apache2 are the same application - just that some Linux distributions refer to it differently within package managers and config files. RedHat-based distros (CentOS, Fedora) refer to it as httpd while Debian-based distros (Ubuntu) refer to it as apache. The commands below use httpd, but if that fails you should try replacing it with apache2

1. Open Apache configuration file. ex /usr/local/etc/httpd/httpd.conf
   • If you do not know where it is you can run httpd -t -D DUMP_INCLUDES to output the locations. In rare cases, if multiple versions or instances of apache exist on the machine the httpd executable may not output the correct information. Using where httpd can potentially show multiple executables you may explicitly try /usr/local/bin/httpd -t -D DUMP_INCLUDES
2. Edit or add the file to include LimitRequestLine 40000 and LimitRequestFieldSize 40000
3. Restart apache httpd restart or apache2 restart

The limit of 40,000 is arbitrary and can be adjusted. We recommend you use a this value or even 20,000 to help catch progromatic error you may cause. A common issues occurs when using MySQL IN Clause, which can have its own limits, to filter out data already slected and cached in the ui/frontend. As example, passing ten thousand ID's is not a good solution and we might want that to actually fail so we can write a better MySQL join and paginate properly. The C6 Frontend bindings have built in pagination support. Checkout all our documentation out at CarbonORM.dev.

Locate NGINX Configuration File: The main NGINX configuration file is typically located at /etc/nginx/nginx.conf. However, the specific file to edit can vary depending on your server setup. It might be in a site-specific configuration file in the /etc/nginx/sites-available/ directory.

Edit the Configuration File:

Open the configuration file in a text editor. For example, using nano:

sudo nano /etc/nginx/nginx.conf

Find the http block in the configuration file. Inside this block, add or modify the large_client_header_buffers directive. For a 20,000 character URL limit, you need to set the buffer size larger than 20,000. The directive takes two parameters: the number of buffers and the size of each buffer. For example:

http {
    ...
    large_client_header_buffers 4 24k;
    ...
}

Here, 4 is the number of buffers, and 24k is the size of each buffer. This will roughly equal 20,000 character limit. To move to 80,000 change the directive to large_client_header_buffers 4 100k;. Adjust the size as needed to accommodate your URL length requirement. Test the Configuration: After editing, it's important to test the configuration for syntax errors:

sudo nginx -t

Reload NGINX: If the configuration test is successful, reload NGINX to apply the changes:

sudo systemctl reload nginx

Verify Changes: Test with a long URL to ensure the new configuration is working as expected.

You can refer to this StackOverflow post.

Our middleware ORM's do not require a browser. The CarbonReact and Javascript bindings send full JSON SQL with does require large request URI's. This is typically not an issue for most applications, but may require a few custom configurations to your CDN and Apache/Nginx/etc server. This also means, for now, we genearlly do not support Microsoft Edge browser. A plan to use WebSockets to optionally facillitate requests would allow Edge support, but no timeline has been set. Many small simple select statments will work. When a Select gets complex with joins you generally will see anywhere from 2,000 - 12,000 characters being sent to the backend system. Queries may be even longer and C6 defines no limit on its own.

CDNs also impose limits on URI length, and will return a 414 Too long request when these limits are reached. Generally, C6 Rest based requests will/should not be cached on a CDN level and should provide no indexing to search engines. This is a general precident for all web based API's. Your applications rest endpoint defaults to /rest/ but may be customized with a prefix. This endpoint should be blocked from caching.

Here is an open-source example from my time at Drop-In Gaming. Though full examples are available at the associated links, the following are stripped of imports. getGameList.tsx

export default restApi<{}, iDig_Chat_Global_Messages, {}, iGetC6RestResponse<iDig_Chat_Global_Messages>>({
    axios: DropVariables.axios,
    tableName: C6.dig_chat_global_messages.TABLE_NAME,
    requestMethod: GET,
    queryCallback: {
        [C6.SELECT]: [
            dig_chat_global_messages.GLOBAL_MESSAGE_ID,
            dig_chat_global_messages.GLOBAL_MESSAGE_FROM_USER,
            dig_chat_global_messages.GLOBAL_MESSAGE,
            dig_chat_global_messages.GLOBAL_MESSAGE_DATETIME,
            dig_chat_global_messages.GLOBAL_MESSAGE_ADVERTISEMENT,
            dig_chat_advertisements.ADVERTISEMENT_ACTIVE,
            dig_chat_advertisements.LAST_SHOWN_AT,
            [um_metadata.UM_VALUE, C6.AS, 'profile_photo'],
            users.DISPLAY_NAME,
            users.USER_NICENAME
        ],
        [C6.JOIN]: {
            [C6.INNER]: {
                [users.TABLE_NAME]: {
                    [dig_chat_global_messages.GLOBAL_MESSAGE_FROM_USER]: users.ID
                }
            },
            [C6.LEFT_OUTER]: {
                [um_metadata.TABLE_NAME]: {
                    [um_metadata.USER_ID]: users.ID,
                    [um_metadata.UM_KEY]: 'profile_photo',
                },
                [dig_chat_advertisements.TABLE_NAME]: {
                    [dig_chat_advertisements.ADVERTISEMENT_ID]: dig_chat_global_messages.GLOBAL_MESSAGE_ADVERTISEMENT
                }
            }
        },
        [C6.GROUP_BY]: [
            dig_chat_global_messages.GLOBAL_MESSAGE_ID,
            um_metadata.UM_VALUE
        ],
        [C6.PAGINATION]: {
            [C6.ORDER]: {
                [dig_chat_global_messages.GLOBAL_MESSAGE_DATETIME]: C6.DESC
            },
            [C6.PAGE]: 1,
            [C6.LIMIT]: 100
        },
    },
    responseCallback: response => {

        const restData = response?.data?.rest A|| [];

        DigApi.digApi.setState((previousState) => ({
            globalMessages: [...previousState.globalMessages || [], ...restData],
        }));

    }
})

Its associated JEST test is even simpler. getGameList.test.tsx

describe('Get Games List', () => {

    test('Asset Game List Not Empty', async () => {

        let bootstrap: DigApi = DigApi.digApi;

        // @link https://reactjs.org/link/wrap-tests-with-act
        await act(async () => {

            /* fire events that update state */
            await getGameList();

            expect(bootstrap.state.backendThrowable).toEqual([]);

            expect(bootstrap.state.games).not.toBeNull();

            expect(bootstrap.state.games).not.toBeUndefined();

        });

        expect(bootstrap.state.games?.length).toBeGreaterThan(12)

    }, 31000);

});

See the full example here.

deleteFeatureGroupReference.tsx

export default restApi<{
    featureId: number,
    groupId: number
}, iDig_Feature_Group_References, {}, iDeleteC6RestResponse>({
    axios: DropVariables.axios,
    tableName: C6.dig_feature_group_references.TABLE_NAME,
    requestMethod: DELETE,
    queryCallback: (request) => {
        request.success = 'Successfully deleted the feature from the group.'
        request.error = 'Failed to remove the feature from the group. Please try again later.'
        return {
            [C6.WHERE]: {
                [C6.dig_feature_group_references.FEATURE_ENTITY_ID]: request.featureId,
                [C6.dig_feature_group_references.GROUP_ENTITY_ID]: request.groupId,
            }
        }
    },
    responseCallback: (_response, request, id) => {

        id && updateRestfulObjectArray<iGroups>((previousState) => {

            const fullGroup: iGroups | undefined = previousState.groups?.find((group: iGroups) => group.entity_id === request.groupId);

            const fullFeature: iDig_Features | undefined = previousState.features?.find((feature: iDig_Features) => feature.feature_entity_id === request.featureId);

            const index : number|undefined = fullGroup?.feature_code?.indexOf(fullFeature?.feature_code ?? '');

            if (index && index > -1) { // only splice array when item is found
                fullGroup?.feature_code?.splice(index, 1); // 2nd parameter means remove one item only
            }

            return fullGroup ? [fullGroup] : [];

        }, "entity_id", "groups")

    }
});

The code directly blow is executing a change using the requestMethod: PUT,.

await putUser({
    ID: myUser?.id,
    user_email: '[email protected]'
})

The full code blow can be found here. putUser.tsx

export default restApi<{ ID: number }, iUsers,{}, iPutC6RestResponse<iUsers>>({
    axios: DropVariables.axios,
    tableName: C6.users.TABLE_NAME,
    requestMethod: PUT,
    queryCallback: (request) => {
        request.success = 'Successfully updated user information!'
        request.error = 'An error occurred while updating user information!'
        return request
    },
    responseCallback: (response, request) => {

        console.log('response from', response)

        if (undefined !== request.user_pass) {

            return;

        }

        updateRestfulObjectArray<iUsers>([removeInvalidKeys(request, [users])], "ID", "users");

    }

});

The registration example.

postUser({
    user_login: userSignUpAndInInformation.user_login,
    user_email: userSignUpAndInInformation.user_email,
    user_birthday: dateOfBirth,
    organization_id: organization_id,
    success: () => {
        return 'Account created!'  // this will appear in a toast notification
    }
}).then(axiosResponse => console.log(axiosResponse))
  .error(error => console.log(error))

The promise returned by the example above does not need to be awaited as the responseCallback in the code below is usually responcible for handing the initial parsing and placed into state.

The gerneral code below.

export default restApi<iPostUser, iUsers & iDig_Temp_Invite & iDig_Parent_User & iDig_User_Info & iDig_User_Organizations, {}, iPostC6RestResponse<iUsers>>({
    axios: DropVariables.axios,
    tableName: [C6.users.TABLE_NAME, C6.dig_temp_invite.TABLE_NAME, C6.dig_parent_user.TABLE_NAME, C6.dig_user_info.TABLE_NAME, C6.dig_user_organizations.TABLE_NAME, ],
    requestMethod: POST,
    queryCallback: (request) => {
        request.success ??= 'Successfully created your account! You may now log in using your new credentials.'
        request.error = 'An unknown issue occurred while creating your account.'
        return request
    },
    responseCallback: (_response, request, id) => {
        // ... responseCallback code here ...
    }

});

Remember that the business logic is shared; all that is needed is class (object) files with the necessary column data; usually presented as constants. We provide an array with a breakdown of types, foreign key references, and column-specific information. Our editable generations include these class variables: a map for column => regex validation, a complex map used for assigning callable functions to column/validation rest query hooks, and finally, a simple array of callables filled with complex SQL database update procedures.

1. We generate a Data Definition Language (DDL) file using common tools like mysqldump.
2. Parse this data using regex or string manipulation and output the data types for JavaScript/Typescript and the Middleware language.
3. Profit.

You can view a generated table file (carbon_users) which includes custom user methods that will persist across regeneration at my open-source php starter template CarbonORM/CarbonReactBootstrap.