Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pass DATASOURCE_DBNAME from Configuration() to MongoDBConnection() #294

Merged
merged 2 commits into from
Jun 30, 2024

Conversation

oh2fih
Copy link
Collaborator

@oh2fih oh2fih commented Jun 29, 2024

  • MongoDBConnection() always uses its default (database: str = "cvedb")
    unless the setting is passed via _datasource_connection_details.
  • Fix typo; connnection with three n's.
  • Test database init:
    • should populate database with name from CVE-Search configuration
    • CVE-Search should be able to use it.

@oh2fih oh2fih force-pushed the configurable-mongodb-database branch from 51f9069 to 0f8ebdf Compare June 29, 2024 17:36
oh2fih added 2 commits June 29, 2024 20:38
MongoDBConnection() always uses its default (database: str = "cvedb")
unless the setting is passed via _datasource_connection_details.
@oh2fih oh2fih force-pushed the configurable-mongodb-database branch from 0f8ebdf to 2a7e6c5 Compare June 29, 2024 17:39
@oh2fih
Copy link
Collaborator Author

oh2fih commented Jun 30, 2024

Database population seems to work, but there's somewhat less data, likely for reasons unrelated to this change.

As it is the cpe and cves that don't match it's likely there were some problems with the NVD API; trying to re-populate.

show dbs

admin      40.00 KiB
config    108.00 KiB
cvedb       3.48 GiB
cvedbfoo    1.95 GiB
local      80.00 KiB

db.schema.find().count()

collection cvedb cvedbfoo
capec 559 559 ⚖️
cpe 1 211 052 817 764 ⚠️
cpeother 0 0 ⚖️
cves 241 201 189 192 ⚠️
cwe 1 312 1 312 ⚖️
info 6 6 ⚖️
mgmt_blacklist 0 0 ⚖️
mgmt_users 0 N/A
mgmt_whitelist 0 ⚖️
schema 1 1 ⚖️
via4 148 300 148 300 ⚖️

db.collection.totalSize()

collection cvedb cvedbfoo
capec 704 512 704 512 ⚖️
cpe 410 382 336 265 330 688 ⚠️
cpeother 12 288 12 288 ⚖️
cves 3 272 286 208 1 779 433 472 ⚠️
cwe 421 888 421 888 ⚖️
info 73 728 73 728 ⚖️
mgmt_blacklist 12 288 12 288 ⚖️
mgmt_users 61 440 N/A
mgmt_whitelist 12 288 12 288 ⚖️
schema 57 344 40 960 ⚠️
via4 50 552 832 50 655 232 ⚖️

@oh2fih
Copy link
Collaborator Author

oh2fih commented Jun 30, 2024

Database population seems to work, but there's somewhat less data, likely for reasons unrelated to this change.

As it is the cpe and cves that don't match it's likely there were some problems with the NVD API; trying to re-populate.

It seems clear this is not related to this patch at all. I just got a bunch of these:

- CveXplore.core.database_maintenance.sources_process - ERROR - Retrieval of api data resulted in an Exception: Timeout on reading data from socket...

(I still had the Squid proxy from testing cve-search/cve-search#1102 in front of the API. Looks more promising without it.)

@P-T-I P-T-I merged commit 1ccd661 into cve-search:master Jun 30, 2024
3 checks passed
@oh2fih oh2fih deleted the configurable-mongodb-database branch June 30, 2024 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants