Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite #140

Closed
wants to merge 48 commits into from
Closed

Rewrite #140

wants to merge 48 commits into from

Conversation

cdoublev
Copy link

@cdoublev cdoublev commented May 10, 2021

Please follow the commits to review this pull request. You can check that tests pass for each one, or that a commit that adds a new test is fixed by the next commit. You may have to reload the test script and/or reinstall dependencies.

Other fixed issues (non-exhaustive):

  • numbers with an exponent (as well as for other numeric types)
    • 1e1 and 1e+1 should be 10 instead of undefined
    • 1e-1 should be 0.1 instead of undefined
  • length/percentage without leading 0
    • .10unit should be 0.1unit instead of 0.1unit
  • dimensions with uppercase unit:
    • 1Unit should be 1unit instead of undefined
  • angle
    • 0 should be 0deg instead of undefined
  • empty resource
    • url(), url( ), url(""), should be undefined instead of url()
  • resource with invalid escape sequence
    • url(val\\\nid.url) should be undefined instead of url(val\\\nid.url)
  • resource with valid escape sequence
    • url(file\\0 \\g.jpg) should be url(file\\0\\g.jpg) instead of undefined
  • resource resolved with double quotes
    • url(file.jpg) and url('file.jpg') should be url("file.jpg") instead of url(file.jpg)
  • non escaped newline in string
    • "\n" should be undefined instead of "\n"
  • string resolved with double quotes
    • 'string' should be "string" instead of 'string'
  • color
    • #1234567 should be undefined instead of rgb(18, 52, 86)
    • #ffffffff should be rgb(255, 255, 255) instead of rgba(255, 255, 255, 1)
    • RED should be red instead of RED
    • RGb(0, 0, 0) should be rgb(0, 0, 0) instead of undefined
    • rgb(300, 300, 300, 2) should be rgb(255, 255, 255) instead of undefined
    • rgb(-1, -1, -1, -1) should be rgba(0, 0, 0, 0) instead of undefined
    • hsl(540, 100%, 50%) should be rgb(0, 255, 255) instead of rgb(0, 0, 0)
    • hsla(400, 200%, 200%, 200%) should be rgb(255, 255, 255) instead of undefined
    • hsla(-1, -1%, -1%, -1%) should be rgba(0, 0, 0, 0) instead of undefined
    • rgba(245.5, 245.5, 0, 50.1%) should be rgba(246, 246, 0, 0.5) instead of undefined
    • rgba(0, 0, 0, 49.9%) should be rgba(0, 0, 0, 0.498) instead of undefined
    • rgba(5%, 10%, 20%, 0.4) should be rgba(13, 26, 51, 0.4) instead of rgba(12, 25, 51, 0.4)
    • rgb(33%, 34%, 33%) should be rgb(84, 87, 84) instead of rgb(84, 86, 84)
  • background-position (as well as for color gradient functions that use <position>)
    • side should be undefined instead of side
    • 1 should be undefined instead of 1
    • left left should be undefined instead of left left
    • top top should be undefined instead of top top
    • left top center should be undefined instead of left top center
    • left top 50% should be undefined instead of left top 50%
    • top 50% should be undefined instead of top 50%
    • 50% left should be undefined instead of 50% left
    • 0 0 should be 0px 0px instead of 0 0
    • 0% should be 0% center instead of 0%
    • left should be left center instead of left
    • top should be center top instead of top
    • center should be center center instead of center
    • top left should be left top instead of top left

@codecov-commenter
Copy link

codecov-commenter commented May 11, 2021

Codecov Report

Merging #140 (ae7da59) into master (b527ed7) will increase coverage by 51.16%.
The diff coverage is 91.61%.

❗ Current head ae7da59 differs from pull request most recent head 3ae3298. Consider uploading reports for the commit 3ae3298 to get more accurate results
Impacted file tree graph

@@             Coverage Diff             @@
##           master     #140       +/-   ##
===========================================
+ Coverage   37.39%   88.55%   +51.16%     
===========================================
  Files          87       95        +8     
  Lines        1182     2001      +819     
  Branches      227      465      +238     
===========================================
+ Hits          442     1772     +1330     
+ Misses        633      200      -433     
+ Partials      107       29       -78     
Impacted Files Coverage Δ
lib/allExtraProperties.js 100.00% <ø> (ø)
lib/properties/azimuth.js 4.44% <2.32%> (+4.44%) ⬆️
lib/properties/fontWeight.js 21.42% <16.66%> (+21.42%) ⬆️
lib/properties/borderCollapse.js 33.33% <25.00%> (+33.33%) ⬆️
lib/properties/fontStyle.js 33.33% <28.57%> (+33.33%) ⬆️
lib/properties/fontVariant.js 33.33% <28.57%> (+33.33%) ⬆️
lib/properties/lineHeight.js 33.33% <28.57%> (+33.33%) ⬆️
lib/properties/borderSpacing.js 69.23% <66.66%> (+69.23%) ⬆️
lib/properties/font.js 72.72% <70.00%> (+72.72%) ⬆️
lib/properties/clip.js 88.46% <78.57%> (+88.46%) ⬆️
... and 100 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b527ed7...3ae3298. Read the comment docs.

@cdoublev cdoublev force-pushed the rewrite branch 2 times, most recently from b0c39bb to 363fd1d Compare May 13, 2021 09:24
This was referenced May 13, 2021
@cdoublev cdoublev marked this pull request as ready for review May 13, 2021 09:31
@cdoublev cdoublev force-pushed the rewrite branch 6 times, most recently from 1986e57 to d51e8f0 Compare May 15, 2021 16:43
@cdoublev cdoublev force-pushed the rewrite branch 2 times, most recently from 33adb04 to 57799cc Compare May 23, 2021 16:19
@cdoublev cdoublev force-pushed the rewrite branch 2 times, most recently from cd66949 to feebfaa Compare June 10, 2021 19:15
@cdoublev cdoublev force-pushed the rewrite branch 2 times, most recently from 9878bec to 3ae3298 Compare June 11, 2021 07:51
cdoublev added a commit to cdoublev/jsdom that referenced this pull request Jun 11, 2021
@ghost
Copy link

ghost commented Nov 29, 2021

@cdoublev: I’m not a JSDOM maintainer or anything, but I just really wanted to say that I really appreciate your effort into this PR!

I have decided to take a read on your changes, and I have been thinking: Wouldn’t it make more sense to parse the values into some kind of AST? And the apply some form of normalization to them depending on the property name.

You might even be able to use something like the CSS typed OM for it. I think it’d be neat to have something like:

string → list of tokens → CSS typed OM → normalized CSS typed OM

I feel like that’d make more sense than using strings for everything, and it would also make sense to implement the typed OM in JSDOM. (Though of course the typed OM would need to be implemented in JavaScript. There is a library, but it doesn’t seem accurate enough, and also seems abandoned.)

@cdoublev
Copy link
Author

It would have made even more sense to rewrite the whole package and implement the procedures from CSS OM, CSS Syntax, and CSS Typed OM, indeed... which is what I did (95% done). It was a lot more work than I expected, but it is absolutely necessary eg. for parsing the new CSS math functions, backtracking while parsing some complex grammars like <position>, etc.

A few days ago, I started thinking about how to get the best chances to get a rewrite of CSSStyleDeclaration accepted on the JSDOM repo, but I'm afraid that this would be a task even harder than all what I already did... :)

Anyway, I should also note that I'm aware that some things are wrong in this PR. My plan was to close it after the work on my rewrite would be done (max. 2 weeks), and to share a link to the corresponding repo so that people can use it for their own needs (JSDOM or any other context).

@domenic
Copy link
Member

domenic commented Nov 29, 2021

Hey folks, dropping in here as, I guess, the main jsdom maintainer, to give some background and ideas on the path forward.

jsdom's CSS processing has always been offloaded to other packages, maintained outside the "core team" by various folks. The layering has always been a bit scattered, with a bit of code living in jsdom, a chunk in cssom, and a chunk in cssstyle. (We even had an attempted from-scratch rewrite at https://github.com/Zirro/css-object-model which never got finished.) The main constraint is maintainers.

A while back, primary maintainership of cssstyle transitioned from chad3814 to jsakas. As part of that we moved the repository into this organization. It looks like recently jsakas hasn't had as much time to review and merge PRs, which is very understandable---I'm barely keeping up on the main jsdom repo itself, doing work in weekend spurts. It's a hard job. So that resulted in various ambitious PRs, like this one and #116, not getting merged.

So, where do we go from here? Well, I guess the realization I'm having is that the reason we transitioned the repo into the jsdom org, is to give the jsdom team the ability to appoint new maintainers. And given the intense effort being invested by @cdoublev, I'd love to nominate them, if they accept. That is, give them write permissions to this repository, and let them be the new person driving the cssstyle npm package and its jsdom integration.

Here are some concrete ideas for what that would probably look like:

  • @cdoublev would get write and npm publish permissions, and can give anyone else they want to work with those same permissions. They will get full control over the contents of the repository and npm package, although the jsdom core team has some suggestions (see below).
  • Unfortunately, I at least cannot commit to providing code review for cssstyle itself. So this might be a lonely job. (But maybe not; a lot of people seem interested in this thread!)
  • I can provide (in weekend spurts) time to review and merge jsdom integration PRs, where you upgrade jsdom's cssstyle dependency and make any integration changes necessary. In such cases I would trust you to have done the right thing over in cssstyle, and focus the review just on fitting idiomatically into jsdom and ensuring that jsdom tests don't regress (or at least, if there are regressions, they are outweighed by many other new tests passing).
  • Note that specific jsdom issues, like the ones you cite in the OP, wouldn't be fully closed until we have web platform tests verifying them passing. But we could still merge without such tests if necessary; we'd just tag the issues as maybe fixed (add tests to confirm).

Here are some suggestions (not requirements) for how to run the cssstyle repository:

  • As best as possible, have good test coverage, ideally measured and maintained.
  • Even better, figure out a way to make most of your test coverage be via web platform tests. I'm not sure appropriate WPTs exist for this level of CSS stuff, or if they do, how you would run them against cssstyle alone (instead of against jsdom-with-cssstyle). But if you can pull that off, you benefit the entire web ecosystem, including real web browsers and the specs. (We've found several browser interop issues, spec bugs, and test bugs in jsdom using this approach.)
  • Implement as closely to specs as possible. This makes future maintenance, including by other maintainers, much easier. Ideally with comments pointing to spec lines.
  • Use major version bumps liberally. We don't worry about conserving integers in the jsdom GitHub org :).
  • Consider swallowing the cssom package (like https://github.com/Zirro/css-object-model tried to do). Its maintainer is also bandwidth-constrained, and the split doesn't feel that helpful to me.
  • Consider following the pattern shown in jsdom/whatwg-url, or jsdom/domexception, and using webidl2js to generate wrapper classes around the impl. (It looks like this is your plan; yay!)
  • Consider adhering to the jsdom repo "template", e.g. using Yarn, GitHub Actions, ESLint with @domenic/eslint-config as the base, Node v12 minimum, etc. See e.g. Various repo hygeine updates whatwg-mimetype#21 for recent work on one of our packages to align it. (But especially on ESLint style choices, you are the boss---if you prefer a different style, go for it.)
  • Consider avoiding compile-to-JS languages like TypeScript.

Anyway, let me know if this sounds like a good plan, and I'm happy to give you write and publish access.

@cdoublev
Copy link
Author

Thank you Domenic, I'm aware of the personal time investment that JSDOM represents for you. I didn't expect to be read by you here, honestly.

My initial motivation was to make style.width = { toString: () => '100px' } (valid CSS declaration) work in the tests of another project (JS implementation of Web Animations) that are running in a JSDOM environment, therefore to make this declaration work in this library. This led me to discover its issues with "common" CSS syntaxes, then to realize that jsakas does not have anymore the time to manage this library, and also that a complete rewrite is needed to support some CSS syntaxes. I also think that improving CSS support in JSDOM is not a priority for its users, because most often their tests can workaround an unsupported syntax with an alternative syntax.

Despite this, I started from scratch an implementation strictly based on the relevant CSS specs, and I have to say that I spent more time on this task than would have been reasonable. This is the reason why my plan (as mentioned above) was to complete this implementation, publish it on Github under the MIT license, and leave a comment here inviting everyone to make further progress on the goal of improving CSS support in JSDOM.

The goal of this plan for me is to restore my personal time. If no JSDOM user/maintainer takes hold of the "collective" goal, I will invest my personal time again to grasp the integration of WPTs tests, which I look at quite often, which I have already installed but which I failed to run (the result of a brief attempt). Another remaining step indeed seems to consolidate a complete implementation of the CSS interface into a single library, obviously using webidl2js internally.

I already had a brief knowledge of the CSS processing layers in JSDOM because I had to fork it to replace this library by a fork providing support for style.width = { toString: () => '100px' }.

I was also aware of Zirro/css-object-model and its use of CSSTree, which came out of my mind afterwards. I believe it was after I finished the backtracking mechanism of the CSS parser that I get another look at CSSTree. But after spending about 1 hour reviewing it, I concluded that it is great for parsing and transforming CSS, but not for processing it like a user agent should, by identifying longhand property values in a shorthand value, by handling property specific rules, by resolving math functions, default values, etc... maybe this is one of the reasons why Zirro / css-object-model never got finished, and maybe it was this conclusion that got me out of my head CSSTree the first time.

Anyway, I haven't covered all topics but to answer you, I think it would be better to wait until my own rewrite is actually finished, I can measure more precisely the remaining steps, and take a step back on my capacity to handle them as an official maintainer.

@cdoublev
Copy link
Author

cdoublev commented Dec 14, 2021

Already 2 weeks since my previous comment... I just created the Github repository for my rewrite of cssstyle in its near current state, so I'm closing this PR.

By replacing cssstyle now, it would already provide better CSS support in jsdom and allow several issues to be closed, but I think the right time to replace cssstyle will be after some (still, sorry) remaining tasks are done. It is mainly about:

  • 1. Update the version of @webref/css
  • 2. Implement more property/type specific rules
  • 3. Code cleanup
  • 4. Implement the full CSS interface

So why have I published this rewrite now? Because I declared this intent in my previous comment but also because it can now be used by someone able to progress faster than me who wants to use it for its own rewrite.

1. @webref/css

This rewrite creates an abstract syntax tree to represent the grammar of a property whose value is specified as input. The definitions (grammars) of the CSS properties and types are imported from @webref/css.

The main issue is that in specifications, CSS type definitions are not always defined with the CSS syntax or the markup required to be extracted by @webref/css and/or transformed into an abstract syntax tree. Another issue is that some properties and types are defined in different specifications with different values.

Because @webref/css is semi-automatically updated after some changes to the specs are published, it seems safer to define a fixed version number in the dependencies. I am trying to transition to its latest version from a previous version that is only a few months old, and it is not easy.

2. Implement more property/type specific rules

While @webref/css is of great help, the specific rules for parsing and serializing some properties and types require a specific implementation. Several types and properties are already parsed and serialized conforming to these rules: terminal types, <color>, <position>, <gradient>, border, background, margin, etc., but there are many more.

This task seems appropriate for external contributions. Which means that some documentation is needed. Writing this documentation is an underlying task that I did not add to the above list.

3. Code cleanup

Some parts of this rewrite are residues of cssstyle or this PR. There are certainly some parts for which I did not get an opportunity to get a review and find optimizations or issues to be fixed. There are also some other parts resulting from back and forth while implementing a feature and for which a clean up requires a special attention.

4. Implement the full CSS interface

CSS is not limited to CSSStyleDeclaration. I think the implemented procedures represent a significative part of what is required to implement the whole CSS object model (rules, queries, imports, stylesheet, etc.), and doing this before an integration to jsdom could prevent issues that would be otherwise discovered while transitioning from a partial to a full CSS implementation.


Another painfull task on which I worked the last few days was about an issue related to recursive backtracking. NodeJS and most browser vendors do not implement tail call optimization. When a CSS value should be matched against a CSS grammar allowing many combinations, recursive backtracking can lead to exceeding the maximum call stack size. Eg. the grammar for background involves 7! (5040) permutations when all its longhands are specified in input, and 6! + 5! + ... (873) (8659) more when some are omitted (total = 13699). The workaround is a trampoline but matching a value for this property is still very slow. The rewrite does not include them, but I found solutions to optimize parsing these kind of grammars.

I will update this thread while progressing on these tasks.

@cdoublev
Copy link
Author

cdoublev commented Jan 21, 2022

EDIT: nevermind, I found a clean solution to return a CSSStyleSheet() from both parse a stylesheet and parse a CSS stylesheet, and a CSSRule subclass from parse a CSS rule. I hope to finish the implementation with a minimum of unit tests next week, before trying to integrate the WPTs, the ESLint rules mentioned above, forking jsdom and prepare a PR, writing some docs, etc...

Domenic,

I've successfully implemented almost the whole CSS interface, but there is an architectural issue that I'm struggling with since almost 2 weeks and that is partially related to webidl2js.

A CSSStyleSheet is created from the result (CSS rules) of parsing a CSS stylesheet as a string or byte stream. I'm not sure if this result should already be instances of classes inheriting CSSRule or simple representation objects to map to instances of the corresponding classes when creating the CSSStyleSheet, but CSSRule.parentStyleSheet should reference the instance of CSSStyleSheet. Reciprocally, the CSSStyleSheet should reference the CSSRule instances via CSSStyleSheet.cssRules (a getter of CSSRuleList that shares the private list of rules). Finally, CSSStyleSheet.insertRule() appends a new instance of a CSSRule subclass in CSSStyleSheet.cssRules, and the same method may exist for some CSSRule subclasses such as CSSStyleRule or CSSKeyframesRule, which are "intermediary" parent classes of CSSNestingRule or CSSKeyframeRule.

Currently, my implementation of parseCSSStyleSheet(), whose corresponding procedure does not define its return value, does not return a CSSStyleSheet: the user has to run CSSStyleSheet.create(globalThis, undefined, { cssRules: parseCSSStyleSheet() }). I can successfully parse a CSS stylesheet and get the expected tree of CSS objects, but I'm struggling to implement CSSStyleSheet.insertRule() because of circular dependency issues I already had when trying to return a CSSStyleSheet from parseCSSStyleSheet().

Can you see how to create a CSSRule subclass from (the result of) parseCSSRule(), the last statement that is run for CSSStyleSheet.insertRule(), to avoid a circular dependency?

Below is a (extremely) simplified representation of the dependencies between the required parts.

/**
 * 1. Requiring parseCSSStyleSheet() requires ...
 * 2. ... CSSStyleSheet, which requires ...
 * 3. ... CSSStyleSheetImpl, which requires ...
 * 4. ... CSSRuleSubClass, which requires ...
 * 5. ... CSSRuleSubClassImpl, which requires ...
 * 6. ... parseCSSRule(), which requires CSSRuleSubClass (repeat from step 4)
 */

function parseCSSStyleSheet(input) {
  // ... parse input against CSS grammar
  return CSSStyleSheet.create(document, undefined, { cssRules: input })
}

// Note: only the first argument is defined by the corresponding procedure in CSS Syntax
function parseCSSRule(input, parentStyleSheet, parentRule) {
  // ... parse input against CSS grammar
  return CSSRuleSubClass.create(document, undefined, { ...input, parentStyleSheet, parentRule })
}

class CSSRuleSubClassImpl extends CSSRuleImpl {
  constructor(globalObject, args, { parentStyleSheet, parentRule, ...privateData }) {
    this.parentStyleSheet
    this.parentRule
  }
  insertRule(input) {
    this.cssRules.push(parseCSSRule(input), this.parentStyleSheet, this.parentRule)
  }
}

// webidl2js wrapper
const CSSRuleSubClass = {
  create: (...args) => new CSSRuleSubClassImpl(...args)
}

const typeMap = {
  CSSRuleSubClass,
}

class CSSStyleSheetImpl {
  constructor(globalObject, args, { cssRules }) {
    this.cssRules = cssRules.map(rule => 
      typeMap[rule.type].create(globalObject, undefined, { ...rule, parentStyleSheet: this }))
  }
  insertRule(input) {
    this.cssRules.push(parseCSSRule(input, this))
  }
}

// webidl2js wrapper
const CSSStyleSheet= {
  create: (...args) => new CSSStyleSheetImpl(...args)
}

This is the best workaround I got for now:

// cssom.js
module.exports = {
  CSSStyleSheet: require('./cssom/CSSStyleSheet.js'),
  CSSRule: require('./cssom/CSSRule.js'),
  // ...
}

// interface.js
const { CSSStyleSheet, ...cssom } = require('./cssom.js')
function parseCSSStyleSheet(input) {
  // ...
  return CSSStyleSheet.create(document, undefined, { cssom, cssRules: input })
}
function parseCSSRule(input, parentStyleSheet, parentRule) {
  // ...
  return cssom[input.type].create(document, undefined, { ...input, parentStyleSheet, parentRule })
}

// cssom/CSSStylesheet-impl.js
class CSSStyleSheetImpl {
  constructor(globalObject, args, { cssom, cssRules, ...privateData }) {
    // ...
    this.cssRules = cssRules.map(rule => 
      cssom[rule.type].create(globalObject, undefined, { cssom, parentStyleSheet: this, ...rule })
  }
  insertRule(input) {
    this.cssRules.push(parseCSSRule(input, this))
  }
}

// cssom/CSSRuleSubClass-impl.js
class CSSRuleSubClassImpl {
  constructor(globalObject, args, { cssom, parentStyleSheet, ...privateData }) {
    // ...
    this.cssRules = cssRules.map(rule => 
      cssom[rule.type].create(globalObject, undefined, { cssom, parentRule: this, ...rule })
  }
  insertRule(input) {
    this.cssRules.push(parseCSSRule(input, this.parentStyleSheet, this))
  }
}

@cdoublev
Copy link
Author

Hi here,

I'm sorry because:

  • it's been 1 month since my last update
  • it's been (already) 3 months that I work on the implementation of the complete CSS API (in addition to CSSStyleDeclaration)
  • there is still a lot to do

I had quite a hard time with parsing selectors. Domenic may somewhat know what I'm talking about, even if matching selector (Element.querySelector()) is currently handled with nwsapi in jsdom. There are rules such as:

  • & (nesting selector) cannot be used in a style rule appearing at the top level of the stylesheet
  • a namespace prefix used in a selector should have been declared with @namespace
  • a pseudo-element can only be associated to some pseudo-classes
  • some pseudo-classes (is(), has(), nth-child(), etc.) have specific parsing
  • etc

These rules required significant changes, which are visible in this commit.

The W3C team does an amazing job with developing and maintaining all these specs, imho. But some CSS specs still have parts that are rather nebulous, to put it mildly. Many issues are reported on the w3c/csswg-drafts repo, some of which have been around for months or even years. I also reported several issues myself.

The length of my todo list is still pretty demoralizing. This is one of the reason I do not publish it, and it is for this reason that I prefered to push "big" commits like the one referenced above, to spend less time reporting issues on w3c/csswg-drafts, and above all, to avoid announcing a delay that I may not be able to honor.

I am aware that the deprecation date of NodeJS v12 is scheduled for April 22. Aiming for that date gives me time to fix the remaining issues and implement the missing CSS features supported in all browsers. I am thinking in particular of @font-face, which seems rather complex to implement, or of @import, for which I have not yet decided the behavior to implement. Ideally, the referenced style sheet should be fetched, knowing that fetch will soon be supported in NodeJS.

Experimental features or features not implemented or partially implemented in browsers, such as @layer, @custom-query, new color functions, etc (the list is very large), will have to wait.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants