CLI: Import LB3 model(s) into LB4 project [EXPERIMENTAL]

[bajtos]

See #2480

This pull request is adding a new CLI command lb4 import-lb3-models:

• The command requires a path to a LB3 application (either the top-level project directory or the server file).
• After the app is loaded via require(), the user is presented with a list of models to import.
• Important: built-in models like User and AccessToken are included in that list.
• For each selected model, a corresponding src/models/{model-name}.ts file is created.

Known limitations (features intentionally left out of this initial pull request):

• Connector-specific metadata in property definitions is not imported. LB3 model import: preserve connector-specific metadata in property definitions #3810
• When a property is defined with a complex object type, the nested property definitions are not converted from LB3 to LB4 format. LB3 model import: convert definition of nested properties #3811
• Model relations are not imported. Spike: how to import relations from LB3 models #3812
• Models inheriting from application-specific models (including LB3 built-in models like User) cannot be imported yet. Spike: how to import LB3 models extending a custom base model #3813
• For models attached to MongoDB datasource, the imported LB4 model contains incorrect definition of the primary key property of ObjectID type. LB3 model import: handle ObjectID type for MongoDB #3814

/cc @strongloop/loopback-next

Checklist

👉 Read and sign the CLA (Contributor License Agreement) 👈

• npm test passes on your machine
• New tests added or existing tests modified to cover all changes
• Code conforms with the style guide
• API Documentation in code was updated
• Documentation in /docs/site was updated
• Affected artifact templates in packages/cli were updated
• Affected example projects in examples/* were updated

👉 Check out how to submit a PR 👈

[bajtos]

Based on feedback from @raymondfeng and the fact that many important parts of model definitions are contributed by runtime (e.g. PK type and strict mode is decided by the connector), I decided to rework lb4 import-model to load the entire LB3 application and obtain model definitions from runtime metadata.

Screenshot of the new version at work, using our examples/lb3-application/lb3app:

$ lb4 import-model lb3app/server/server.js

**WARNING WARNING WARNING**
This command is experimental and not feature-complete yet.
Watch the progress at https://github.com/strongloop/loopback-next/issues/2480

? Select models to import: CoffeeShop
Model CoffeeShop will be created in src/models/coffee-shop.model.ts

   create src/models/coffee-shop.model.ts
   update src/models/index.ts

Here is the content of generated src/models/coffee-shop.model.ts:

import {Entity, model, property} from '@loopback/repository';

@model({
  settings: {
    strict: false,
    forceId: false,
    base: 'PersistedModel',
    replaceOnPUT: true,
    validateUpsert: true,
    idInjection: true
  }
})
export class CoffeeShop extends Entity {
  @property({
    type: 'number',
    id: 1,
    generated: true,
    updateOnly: false,
  })
  id?: number;

  @property({
    type: 'string',
    required: true,
  })
  name: string;

  @property({
    type: 'string',
  })
  city?: string;

  // Define well-known properties here

  // Indexer property to allow additional data
  // eslint-disable-next-line @typescript-eslint/no-explicit-any
  [prop: string]: any;

  constructor(data?: Partial<CoffeeShop>) {
    super(data);
  }
}

export interface CoffeeShopRelations {
  // describe navigational properties here
}

export type CoffeeShopWithRelations = CoffeeShop & CoffeeShopRelations;

@raymondfeng what do you think, is this closer to the user experience you are envisioning for lb4 import-model?

@strongloop/loopback-next WDYT?

[raymondfeng]

@bajtos Please investigate CI failures.

[bajtos]

@raymondfeng thank you for the initial approval. The pull request was not done yet. Today, I added many new tests and fixed handling of edge cases. Now I consider the pull request ready for final review & landing.

Does it still LGTY?

[agnes512]

I am a bit lost here. In the docs, it says

you can modify your LB3 model to inherit from Model,
PersistedModel or KeyValueModel, then import the model and finally update
the generated model

if a model extends from the list, can it be imported or not?

[bajtos]

if a model extends from the list, can it be imported or not?

TL;DR: These two aspects are orthogonal. If a model extends from Model, PersistedModel or KeyValueModel (all of them are in the list), then it can be imported. If it inherits from a different model class, then it cannot be imported yet.

We have two list of model names:

1. List of model names to hide from the CLI prompt when asking which app models to import. This list is EXCLUDED_MODEL_NAMES plus any model name starting with AnonymousModel_.

   The purpose of this filter is to hide models that cannot be imported (e.g. Model and PersistedModel are built-in in both LB3 and LB4) or which does not make sense to import (e.g. Email).

   The filter does not take into account model inheritance.

   I suppose I could improve the filter to take into account model inheritance. The difficult part is how to explain the user why some models are included in the list and some are not. I would prefer to invest our time into implementing import of models inheriting from other app models instead.

2. List of base models that we know how to handle at LB4 side, see https://github.com/strongloop/loopback-next/blob/42ec9ed2b972fec9ae25b974b65c268a6b2ed34b/packages/cli/generators/import-lb3-model/migrate-model.js#L120-L124

   I created this list by picking built-in LB3 models that have their direct counter-part in LB4.

I'll be opening a new GH issue to implement support for importing models that are inheriting from a model that's not built into LB4. There are two major use cases - import from an LB3 built-in model like User and import from an application-specific model. There are few difficult aspects to figure out, that's why I am leaving this feature out of scope of the initial version.

• How to know where to import the base model from (what path to use in the import statement)?
• How to ensure that the base LB3 model has been either already imported to the LB4 app or is part of the import being in progress?

[agnes512]

Mostly LGTM 👏, I just have one question

[raymondfeng]

Importer for LoopBack 3 models

[bajtos]

@agnes512 @raymondfeng @hacksparrow thank you for the feedback, I pushed a new commit to address your comments.

Does the patch LGTY now?

[bajtos]

One more commit 644d33b - I reworked formatting of "Known limitations" in the docs and added links to follow-up GitHub issues.