feat: API Documentation Generation

[felangel]

Description

As a developer, I want to be able to generate Open API (or similar) documentation for my Dart Frog application so that other developers are able to seamlessly integrate with the API and are aware of the full API specification.

Requirements

• Development Support (Hot Reload Compatibility)
• Production Support
• Open API Specification (or similar)
• Endpoint to view the documentation
• Documentation should be interactive (try it now)

[jaumard]

Hello, there is a requirement I would be interested in. I used to work in Design First. Meaning I first write the openApi definition, then the code. That allow the dev team to all be agree on the API first (and think it through), but it also mean that all validations of models can be applied automatically on routes as they are know at server start (by reading the specs).

Do you think this is something that can be done here?

You can also easily imagine create the routes from the definition directly as path and params are known.

[jakub-stefaniak]

In order to automatically generate OpenAPI specification, we need to take route files as an input and analyze them to see how the developer is handling HTTP methods, identify necessary body fields and etc. . As a non-backend developer, I don't have much experience in this area, but I did some research and found the swagger-autogen package for Express.js. It looks like it could be something that we can look up to. What do you think?

[jakub-stefaniak]

I agree with @GabrielRozendo - #151 (comment) where he pointed that some sort of decoration ( like method annotations ) would be necessary in order to generate OpenAPI flie.

[jaumard]

@jakub-stefaniak we would need decoration for sure, but we should avoid decoration where we can, for example to deduce the return type we can simply modify Response to provide the return type like:

Response<String> onRequest(RequestContext context) {
  return Response(body: 'Welcome to Dart Frog!');
}

For complex type:

Response<MyClass> onRequest(RequestContext context) {
  return Response.json(body: MyClass());
}

Like this pretty easy to generate the openApi, and decorators can be used on class fields and on that method def to define error responses.

[jakub-stefaniak]

Analogous to your solution, we can catch what is needed in request body:

Response<MyClass> onRequest(RequestContext context, UserFormData data) {
  return Response.json(body: MyClass());
}

but UserFormData should be extended with some RequestBody class so we can pass it through Handler.

Handler verifyAuthorizationHeader(
  Handler handler, {
  List<HttpMethod> onMethod = HttpMethod.values,
}) {
  return (context, body) { // RequestContext and RequestBody

    log(body.toString());

    return handler(context, body);
  };
}

What do you think?

[jakub-stefaniak]

Or just add second type to avoid complexity 😅

Response<MyClass, ResponseClass> onRequest(RequestContext context) {
  return Response.json(body: MyClass());
}

[jakub-stefaniak]

I think a good first step in generating an OpenAPI file is to define the OpenAPI class and other models based on the documentation provided by Swagger (https://swagger.io/specification/). It's also important to consider which format we want to use for the OpenAPI file - it can be represented in either JSON or YAML.

[jakub-stefaniak]

Also, we would need some method in dart_frog_gen that will take an index.dart file as an input and return list of method mapped to specific http methods. I was experimenting and came up with this solution:

void getRouteMethods() async {
  final fileUri = Uri.file(
    r'D:\dart_projects\dart_frog\packages\dart_frog_gen\routes\test\index.dart',
  );

  final isolateMirror = await currentMirrorSystem().isolate.loadUri(fileUri);

  final functions = isolateMirror.declarations;

  for (final function in functions.values) {
    Method? method;

    function.metadata.any((m) {
      if (m.reflectee is Method) {
        method = m.reflectee as Method;

        return true;
      }
      return false;
    });

    if (method != null) {
      log('${MirrorSystem.getName(function.simpleName)}: ${method!.method}');
    }
  }
}

it's just a draft, because I just wanted to check if it's possible. I haven't really worked with dart:mirrors before, but it looks like it works.

\routes\test\index.dart content:

import 'dart:io';

import 'package:dart_frog/dart_frog.dart';
import 'package:dart_frog/src/http_method.dart';

class User {}

class UserCreateForm {}

Response<dynamic, dynamic> onRequest(RequestContext context) {
  switch (context.request.method) {
    case HttpMethod.get:
      return _get(context);
    case HttpMethod.post:
      return _post(context);
    case HttpMethod.delete:
    case HttpMethod.head:
    case HttpMethod.options:
    case HttpMethod.patch:
    case HttpMethod.put:
      return Response(statusCode: HttpStatus.methodNotAllowed);
  }
}

@Method(HttpMethod.get)
Response<User, void> _get(RequestContext context) {
  return Response(body: 'Get request');
}

@Method(HttpMethod.post)
Response<User, UserCreateForm> _post(RequestContext context) {
  return Response(body: 'Post request');
}

What do you think? @felangel @jaumard any thoughts?

[abuzar-rasool]

I have tried dart_frog recently and I really want this feature in dart_frog! Let me know @jakub-stefaniak if I can help you with something in this. Thanks!

[jakub-stefaniak]

Hey @abuzar-rasool. I stopped working on this since there was no feedback from authors. Recently @felangel has shown on his twitter working prototype of client package generation. I think API Docs generation will be much easier to implement after this feature release.

[PiotrFLEURY]

Hi everyone !

I tried to resolve this issue using a CLI tool.

I have first version of a Proof Of Concept and I would like to have some feedbacks to know if it is useful to continue in this way or not.

Please welcome nenuphar_cli https://github.com/PiotrFLEURY/nenuphar_cli

This is far to be perfect but it "works" for the moment.
Not all features are adressed.

I tried to follow the Swagger specifications for openapi https://swagger.io/specification/

This CLI used a unpkg version of Swagger UI https://swagger.io/docs/open-source-tools/swagger-ui/usage/installation/

If you want to give a try just start a new Dart Frog project, create a new route (/ path is not considered by the CLI)

Then run the following commands

dart pub global activate --source git https://github.com/PiotrFLEURY/nenuphar_cli.git
mkdir public
nenuphar init
nenuphar gen
dart_frog dev

Visit http://localhost:8080/index.html

You should get a Swagger-ui page like this

What do you think ?
Can it be good to continue and publish this CLI tool ?

Feel free to challenge or give any feedback

Edit

I finally published a first version of the package on pub.dev
https://pub.dev/packages/nenuphar_cli

[fit-jose]

Hi everyone !

I tried to resolve this issue using a CLI tool.

I have first version of a Proof Of Concept and I would like to have some feedbacks to know if it is useful to continue in this way or not.

Please welcome nenuphar_cli https://github.com/PiotrFLEURY/nenuphar_cli

This is far to be perfect but it "works" for the moment. Not all features are adressed.

I tried to follow the Swagger specifications for openapi https://swagger.io/specification/

This CLI used a unpkg version of Swagger UI https://swagger.io/docs/open-source-tools/swagger-ui/usage/installation/

If you want to give a try just start a new Dart Frog project, create a new route (/ path is not considered by the CLI)

Then run the following commands

dart pub global activate --source git https://github.com/PiotrFLEURY/nenuphar_cli.git
mkdir public
nenuphar init
nenuphar gen
dart_frog dev

Visit http://localhost:8080/index.html

You should get a Swagger-ui page like this

What do you think ? Can it be good to continue and publish this CLI tool ?

Feel free to challenge or give any feedback

Edit
I finally published a first version of the package on pub.dev
https://pub.dev/packages/nenuphar_cli

Incredible work, I will likely be using dart frog for a new project, and would love to contribute!

[yayahc]

Hi everyone !

I tried to resolve this issue using a CLI tool.

I have first version of a Proof Of Concept and I would like to have some feedbacks to know if it is useful to continue in this way or not.

Please welcome nenuphar_cli https://github.com/PiotrFLEURY/nenuphar_cli

This is far to be perfect but it "works" for the moment. Not all features are adressed.

I tried to follow the Swagger specifications for openapi https://swagger.io/specification/

This CLI used a unpkg version of Swagger UI https://swagger.io/docs/open-source-tools/swagger-ui/usage/installation/

If you want to give a try just start a new Dart Frog project, create a new route (/ path is not considered by the CLI)

Then run the following commands

dart pub global activate --source git https://github.com/PiotrFLEURY/nenuphar_cli.git
mkdir public
nenuphar init
nenuphar gen
dart_frog dev

Visit http://localhost:8080/index.html

You should get a Swagger-ui page like this

What do you think ? Can it be good to continue and publish this CLI tool ?

Feel free to challenge or give any feedback

Edit
I finally published a first version of the package on pub.dev
https://pub.dev/packages/nenuphar_cli

Good work, but as I use apicurio to design openapis this solution is not for me.
I think I'll continue to wait for a concrete implementation directly in Dart Frog.
But keep in mind that your package works well and is a good alternative for those using swagger.

[dgaedcke]

Do we have an ETA on the roadmap for this feature?
I created another request (now closed) before finding this one

[tomarra]

Do we have an ETA on the roadmap for this feature?

Thanks for asking @dgaedcke. We have talked through this internally at VGV and overall it's actually part of a bigger initiative that we need to take on to be able to handle these kinds of requests going forward which is a Plugin/Codegen system. This is something that has been kicking around as an idea and proposal for a while but we haven't been able to pull the trigger on it quite yet. The goal would be to allow for anyone to then be able to hook into the Dart Frog build system and be able to write their own tooling to solve asks like this.

The good news is that we have most of the design of this work detailed out, the bad news is I don't have an ETA on when we are going to be able to put some dedicated resources on this effort given our client needs that we are balancing at the same time. I'm going to see what we can do to publish the design document in some way so we can get more feedback from the community and see if there may be people that would be willing to help.

[a-wallen]

@tomarra I am willing to help. Is there a Discord that we could be invited to?

[tomarra]

@tomarra I am willing to help. Is there a Discord that we could be invited to?

Hi @a-wallen. We actually prefer to have discussion for these topics in GitHub Issues so there isn't another service/barrier that folks need to go through to help contribute, so we don't have a public Discord. If you're willing to help I think it would be worth seeing if we can maybe get a call setup to understand if the Plugin/Codegen topic.

[a-wallen]

In that case, maybe a doc on how plugins will work would be better? That way it could get posted here.

[eaosny]

Do we have an ETA on the roadmap for this feature?

Thanks for asking @dgaedcke. We have talked through this internally at VGV and overall it's actually part of a bigger initiative that we need to take on to be able to handle these kinds of requests going forward which is a Plugin/Codegen system. This is something that has been kicking around as an idea and proposal for a while but we haven't been able to pull the trigger on it quite yet. The goal would be to allow for anyone to then be able to hook into the Dart Frog build system and be able to write their own tooling to solve asks like this.

The good news is that we have most of the design of this work detailed out, the bad news is I don't have an ETA on when we are going to be able to put some dedicated resources on this effort given our client needs that we are balancing at the same time. I'm going to see what we can do to publish the design document in some way so we can get more feedback from the community and see if there may be people that would be willing to help.

Were looking forward to reviewing the design doc :) 👍

Thank you for investing in the library !