About the future #1239
Comments
|
Echoing the thanks to Felicia! Having built some homebrew solutions for this in the past, wanted to offer help in reimplementation efforts. Anyone else game for this? |
|
I've made some research and asked few friends (pro web dev) about the best technology for a reimplementation. As for me a solution based on nuxt.js / node.js / mysql could work : nuxt (vue.js metaframework) make things easy for the frontend (and there is a lot of documentation too) and node can make the link between the database and the frontend. We would need to copy the database tree. I'll try to make a quick graphic demonstrator if I have some band pass. Any folk interested in a javascript reimplementation, manifest yourself here. |
|
I have quite a lot of experience with Go, which I believe would be an even better choice for the backend. The backend shouldn't take too much time, I guess the UI would take a lot of work to create if it is being rewritten. |
|
I'm a relatively new programmer.... okay so I've really been on/off for like 10 years and never really progressed much but I'm starting to really gain interest in React, NoSQL, docker, and Python (including Flask... never did much with Django). I haven't dug into node much but I'm willing to learn and would be interested in helping get this project going again. I don't know how much help I might be, I may just get in the way, but if this gains any traction, please let me know. I'd love to help. And echoing above, I never had a need for PartKeepr but knew it existed and finally had a need and knew right where to go.... it's a well known project, great job to all the hard work the dev team put in here. |
|
Hello to you all, guys. I just wanted to keep you posted on the most recent status and thoughts we were having. First, it is nice to hear that there are at least some folks out there that are willing to help with the problem. From your posts, I see at least experience in NodeJS, Go, and Python. That is good! It is also good that there are still users interested in the project. That means it is not dead. Currently, the development has stalled. There were a few internal discussions going on how to proceed. Let me give you an overview. Update by external resourcesWe had the idea to use a crowdfunding project to collect some money and let a pro do the Symfon 2 -> Symfony 5 migration. We had contacted a pro developer and have already a first quote that might be manageable if a few dozen people participated. We are also trying to get a second offer at least for comparison. Once we have them, we wanted to reach out to you as a community but also to other communities in the hacker scene hoping to raise the required funds. This would mean a bigger PR campaign that would be needed. The problem here is that we do not have a clear communication strategy for the community in general: The IRC channel is mostly empty (only a few people are there hanging around). In the google list, there are around 150 members. I suspect there are many more users out there that might benefit from installing and be willing to help. If you have any good ideas on how to mobilize as many users as possible, feel free to speak up. Discussions about rewritingWe also thought about what could be done to
One thing to identify is that the frontend (what you see in the browser) and the backend (the connection between HTTP and the database) are in fact separate projects/issues. Felicia has defined a common interface between those two that is working great at the moment. This does not mean we have to stick completely with it. Adoptions/corrections/additions might be needed once development progresses. But I'd rather avoid building a complete architecture from scratch. That means we should stick for now with the API designed by Felicita. Frontend updateAs mentioned earlier, one could think of updating the frontend with some other framework. The currently used Ext.js might or might not be "the best" solution. There are for sure alternatives like VueJS/Nuxt.JS that might be more modern. Also, one could think of updating the building environment to e.g. webpack (or vite in case of Vue). When looking at the most pressing problems, the Frontend seems mostly unconcerned as far as I see it. As long as we keep it consistent with the defined API, any updates can be done and even a parallel (call it modern) frontend might be possible. Backend updateHere comes the big issue at the moment. We all agree that the current Symfony 2 is too outdated to be maintained anymore. Nothing is going around that. We know it. The update of three Symfony versions is hard for a newcomer. This is especially true, as some backward-incompatible changes were introduced and the old documentation is only partially available nowadays anymore. So, you need, simply speaking, an expert in the field that knows the bells and whistles that can be used there. If the update was not possible (lacking manpower and no intention to use external resources, see above), we have to think of replacing the backend. I am not telling you to rewrite from scratch. I am telling that the shiny new backend would have two interfaces to comply with: The database (including its current structure) and the API towards the frontend. If we managed to get this running, we could drag&drop replace the current backend with something else. If we do that we can migrate easily and even do a parallel mode similar to the frontend. This "something else" might be written in different languages: NodeJS, Go, Python, Java, Brainfuck, or even PHP with Symfony 5 😄. Which language to use is (at least partly) not a trivial decision. Some languages are better suited, some are worse. The most pressing thing here is: We want to avoid focusing on one person that might drop out for whatever reason and let the same effect happen again we are facing at the moment. For example, if we have only one Java developer, that would be a rather bad choice. I had recently a video call from a company from Germany that was intending to hack a bit on the database of Partkeepr. They wanted to have some information and were not aware that it was possible to replace the backend as I just presented. As they have experience with Python Django, they are intending to build their own backend for their needs. If we could use a cooperation partner as a bigger player like a company (in contrast to free-time contributors), I think that would be beneficial as the timing requirements are completely different. Before you get too excited that you might get your favorite language (which you invited during your studies of computer science or learned somewhere else): If we decide to change the language, a few considerations must be passed! Here is an incomplete list of what comes directly to my mind:
You see, this is only possible with probably a group of developers that are committing themselves to the project in the long term. There needs enough experience to overcome some shortages. At the beginning of the rewrite huge amount of code need to be written and tested. How do we go on from here?I highly recommend that we do not start to split up our community even further and have different versions of the backend running with different frameworks/languages/... This will not help the project at all. We are all looking for a solution fitting best to ourselves, but I think it absolutely essential that we try to work towards a common solution. On the way there we need some objective way to decide what is good or not. Statements like "xxx is best" or "xxx is best because everyone uses it" do us no good in this discussion. We need hard facts. I have my personal preferences as well but for the sake of the project I am willing to participate in an open discussion about what options do we have. Also if such a discussion starts, I would like to take the complete community into account. Maybe there are some hidden talents that could be used? Call for action
|
|
Thanks a lot for the response! As said above, I think for the backend Go would be the best choice:
The biggest disadvantage is however that there are probably a lot less people to program in Go - as you said you need to reach more than a couple of people in order to rewrite and maintain the project. Also Go binaries tend to be a little bit larger, I assume for this project they would be around 30-50MB uncompressed, which is the drawback of not requiring any dependencies. *if no C code is included, which might be required to support sqlite3 |
|
Hey All! Endless apologies for the lack of organization and communication the past 2 years. I had sort of "inherited" the project with admin access to this repository from @Drachenkaetzchen Together with @christianlupus (thnx for the long post Chris! I fully agree to all points you made) and some other community "power user" members we initially got some work done to organize the project with some minimal improvements, mostly on the Issue tracking and CI side of things. Getting some bugfixes merged, etc. Of course then the pandemic hit and everybody their lives where turned upside down. So here we are. What can we do to keep this project alive? Do we keep brute-forcing dependency upgrades as in https://github.com/partkeepr/PartKeepr/projects/3 and #1214 ? We can also try to find a way to replace the backend of the project on top of the current DB schema, but then the question is also if we should focus on "emulating" the current API? If we're going to replace the stack, then I would actually prefer to move towards an OpenAPI Spec so that we can at least build in interoperability from the start. And alternate frontends could be created then too. However the current UI and API is so feature-complete. I don't know how easily we could instead have the current FE/ext.js converted to another API interface. Some better, interoperable, separation between the two would certainly be a win I think. Personally I'm coming from mostly a Python (and Lua/OpenResty/Nginx) background and can't help support any PHP, Node or Go development. However I can of course help with testing, time permitting. For me I would then lean towards a framework like FastAPI. It implements the OpenAPI stack, simple, fast, with extensive typing and self-documenting. Anyone that wants to help with that hit me up. Any suggestions on how to organize this are welcome. I'm certainly willing to add some more people access to the repository, to organize issues, and we can add some more projects to track things: https://github.com/partkeepr/PartKeepr/projects Also come on IRC just to chat with other users if you want. Should be possible to add a matrix bridge as well. [Edit: I've pinned the topic. Maybe we should move this towards a "discussion" format. What do you think?] |
dromer
added
the
meta
|
@Forceu I really appreciate Go projects like Gitea btw! Single distributable binary that just contains pretty much everything you need. |
|
Hmm, since the spec is based on Hydra, maybe converting the API to https://github.com/HTTP-APIs/hydrus (Python3/Flask/Sqlalchemy based) might be the easiest, for the Python heads? The idea of having PartKeepr with semantic web capabilities is interesting. Imagine being able to link different "departments" or "communities" with access to certain parts of the api. RDF based specs are sometimes a bit hard to read, but it can be damn useful in distributed resources on the internet. |
|
Please note that the frontend relies on entity information generated by the backend. The API and DB Schema are also generated by the entity information. The idea was to have one specific place where to define how an object looks like and how it's relations are - which are the entities. Everything else is derived from that. I strongly advice against replacing the backend. If you want to do that, you basically need to re-write everything. PartKeepr was designed to allow filtering and sorting for every field and every relation, and there are no comparable projects that I know of. I also disagree that PartKeepr should be re-written in a "faster" language. The demo system on https://demo.partkeepr.org runs on a low-end VM with little memory, and it's still fast despite the amount of data. If you need something faster it'll come at the cost of dropping the generic query system which can filter and sort everything, and you need to start to use indexes. A "faster" programming language won't help. A PartKeepr installation isn't used by thousands of users simultaneously. The project took me 6 years to get it to the state where I considered it somewhat feature-complete for my requirements, and I did it with a clear vision what it should do and how it should do it. Replacing the backend or the frontend will easily get you over these 6 years I spent on that. What this project needs is people willing to dig into the existing code, understand what it does, and go on from there. |
|
@Drachenkaetzchen Thanks a lot for your insight and of course a big thanks for this great project! It has helped me a lot to organise all my electronics. You are right, the language does not have to be fast, and the search is indeed very convenient and quick. In the end I think you are right that a rewrite will take a lot of effort and it would be more efficient to update deprecated parts of the code. Out of interest, why not use indexes? |
You can't have an index for every field combination a user potentially wants to search. |
|
This might be becoming off-topic now, but couldn't you use a word-list? Basically for every unique word in any category there is an entry that links to the item IDs - that way you would have a very fast search and can have custom categories as well. |
I never had the need for that. PartKeepr's search speed was always fast enough. Let me quote myself:
How you would do it in this hypothetical scenario is then up to you. |
|
The last time I implemented something like this homebrew I actually used
google sheets as a backend, but they removed part of their scripts API for
that.
Ever heard of Joplin? It's written on electron in js/ts and syncs to
whatever db you set. I really like the idea of making this db agnostic to a
degree and allowing users to host their data however they want as long as
there's a plugin / parser between our API and whatever storage.
…On Fri, Apr 15, 2022 at 1:49 PM Felicia Hummel ***@***.***> wrote:
This might be becoming off-topic now, but couldn't you use a word-list?
Basically for every unique word in any category there is an entry that
links to the item IDs - that way you would have a very fast search and can
have a custom categories as well.
I never had the need for that. PartKeepr's search speed was always fast
enough. Let me quote myself:
If you need something faster it'll come at the cost of dropping the
generic query system which can filter and sort everything, and you need to
start to use indexes.
How you would do it in this hypothetical scenario is then up to you.
—
Reply to this email directly, view it on GitHub
<#1239 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADMMNXAC6GUYUTNGEOZVPMTVFHI3XANCNFSM5QVS5YYA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
And in the case of "give me all resistors with a resistance > 400 and resistance < 500 in the package 0603" a word list won't do.
|
|
Yes, that makes sense. I did not realise that was possible with Partkeepr. |
Partkeepr should have its own independent database, I don't think it would make sense to use a 3rd party database. |
|
Modern PHP is actually quite fast. Getting current PK codebase to that modern state though .. needs some many months of dedicated work and I'd rather see a team than a single person take on such a role. However, even if we outsource/crowdfund such an endeavor we would still end up with a highly complex code-base that nobody can, or wants to maintain. I'm willing to donate 3-figures to have someone, or a team, work on something like this. But how do we make sure that there is a core team that will do all the required maintenance and upkeep? |
|
+1 ping to follow progress. not mainly a web dev, can help with testing / small bug fixes though! |
|
Even though I am just a hobbyist user, I believe PartKeepr has a future and would be willing to donate ~1000€ for a continued/restarted development effort (or smaller regular amounts). I sincerely believe PartKeepr deserves it.
That is also my biggest concern. Though hopefully by moving to a current technology stack, there is a larger pool of people with that kind of experience, reducing the complexity of finding a developer. |
|
I agree that funding to support at least the latest version of symphony is mandatory and could represent the least expensive (money, time) option. We first need to find a web develloper that will accept the deal and gives us an estimate in hours. We will then need to raise the funds. To everyone here interested in funding, tell us how much you can give the project for a full symphony 6 upgrade. I'll be giving ~50€. |
|
I am also a hobbyist and currently looking for a part database. I looked on many free and even commercial products, but partkeepr (even in its current state) is exactly what I am looking for and would be my choice #1. Most probably I will start using it for my home use, hopefully there are not too many bugs I have to deal with (I can correct small ones as I have some experience with php / python / c, but I am far away from being a real developer as I took the admin road in IT business). However, I would like to see this project being renovated / resurrected. I would be willing to donate also ~50 EUR, which isn't too much when you have a look on commercial solutions with monthly subscription fees. Thanks for the great work so far. |
|
I've no experience which platform gets the most traffic from web devs - maybe someone else can chime in and set up a bounty? I'm in with 250EUR myself. |
|
I wouldn't mind to donate money either to upgrade this project since most of the alternatives I found don't suite my needs... |
|
I would donate 100 Euros as well. |
|
@Forceu do you think, that percentage is worth the extra attention we cold gain? @christianlupus do you mind to weigh in on this one? |
|
The 10% is for the amount when the bounty gets awarded. Personally I thought 10% is quite a lot, but I also do not know if there are better alternatives. |
|
This is so good @Lopo ! Are these the OG tests working on your re-implemented modern symfony work? Maybe lets call it "2.0alpha1" for now and start having people work on db migration/testing? What are the min/max requirements for this? Btw, maybe we should start a discussion about this topic. And organize a todo in the projects tab |
|
aaaaaaaand ... it's out ! ;) still can contain lot of stupidities, forgotten code, etc. ... so still lot of work to be done |
|
Thanks! I'd like to test. Is the install same as partkeepr? |
|
Also big thanks from me! I'd be in for testing, but I have no idea how to actually start the server. |
|
actually, there's app:install console command ... when i finish some parts, I'll write it to doc ... or maybe someone other (with better english) can now I'm working on security to be able to use ldap as pswd checker (it's really hard hacking :D ) |
|
@Lopo hmm, so with calling your project Limas you do not intend it to be the succession of PartKeepr? or like a PK2.0? or is this something you are open to? |
|
@dromer The naming it Limas was before the vote and discussion in this thread/issue. And also it was pragmatic - easy search parts that are still not ported or use original URLs/services etc. Depends what it will mean to me ... what brings it to me as advantage, what limitations ... how you and others imagine it should be done and work in future ... |
|
Really nice work, Lopo! I am so excited to try it out.
@Lopo: It would be really nice for a basic installation guide... I have tried to get everything up and running for ~2 hours now, and it is still not running.
Should we discuss installation problems here, or better in issues over at the repo itself? |
|
@mindsolve Limas is still something like a tech-preview, not usable for production use, so creating any doc is not yet on my priority list (at least not high) When I decide that it's stable enough, I'll properly test all steps and write some docs |
|
On 8/6/22 22:35, Pavol Hluchý wrote:
When I decide that it's stable enough, I'll properly test all steps and write some docs
OK thanks.
…--
John Griessen -- not symfony aware
|
|
Hello. I'm a software developer at a small company which uses PartKeepr for some 450 different parts. |
|
On Mon, May 30, 2022 at 3:25 AM Germwalker ***@***.***> wrote:
As Partkeepr is WORKING well for years, at least we must focus (money or
dev time) on the backend update (the frontend is old but it's working
anyway) because there are few security issues that needs to be adressed. If
this issues represent a problem now it will become a larger one in few
years. I want to use partkeepr but i don't want it to be a black hole in my
network.
Maybe we can first make an audit to find security issues then setup a
bounty for each one of them ?
Merging the years back of PR could be a first try.
AIUI the PR issues can NOT be dealt with due to the not only age but
deprecation of Symfony2 (and 3 and 4 and 5).
I am no expert but I would think that if one is finding huge hurdles
in the updating on the back end of some complicated software - - -
well - - - might it not be easier to completely recode same?
I would love to be using Partkeepr but am loath to expend many hours
of working doing all the setup and product entry and connecting the
other bits (scanners and printers and the like) and then finding that
I either get to start all over (due to a then system change) or that
the project actually dies.
I don't want Partkeepr to die but things aren't looking good for any
kind of longevity at this point from where I sit. Partkeepr is an elegant
solution but that solution stopped moving some 8 years ago and in
software lifetime terms - - - - that's not exactly recent.
I would say the question(s) are:
1. can the back end be effectively reprogrammed in anything besides
a php type environment?
1.a. yes - - - - what environments make sense
1.b. no - - - - then its a php type framework
might as well stick with symfony but 6 would be the minimum
(I for one am uncomfortable with the short live-times on the
versions!)
2. is there any kind of steering committee?
2.a. yes - - - - they need to bet busy organizing
2.b. no - - - - this is needed
There's been some questions posed to the users.
I would think its time to get something going. Further foot dragging is
only going to nail the lid on the coffin down even harder - - - - and I
don't think that's what anyone want.
HTH
… Message ID: ***@***.***>
|
I released v0.75 less than 8 years ago, and since your "8 years" 15 versions all the way up to 1.4.0 have been released until I had to stop development for health reasons in 2018. Show a little respect to the developers, especially me, before throwing some arbitrary numbers here. |
|
On Thu, Jul 14, 2022 at 3:18 AM Pavol Hluchý ***@***.***> wrote:
@dromer <https://github.com/dromer> Base (hardest) work is now already
done. Now working on tests and fixing some things, that I missed/forgot to
port or not working as they should.
Frontend is the same (just using ExtJS 7.0.0 + few fixes instead of
6.2.0), DB almost the same (not using FOS bundles, so just 1 users table
and so on ...)
Features should be the same (for now), i have some plans what to add (for
example better security, roles usage)
Of course some functionallities are not working for now:
- Patreon - should I create account ?
- Octopart - changed API, so probably needs complete rewrite
- TipOfTheDay - tied to original PK domain for now
- logo - for now using original PK resources, just favicon is new
- tests - Travis etc.
Install script is only for new install, plan is to create script (console
cmd) for import PK DB, that can be used as part of the install
Wonderful!!!!!!!!!!!!!!!!!!!!!
There's life in the 'ol girl yet!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
These two emails are absolutely wonderful news!!!!!!!!!!!!!
(Sorry for the level of superlatives - - - just was so hoping for something
like this and - - - its happening!!!)
So - - - to test the 'new' PartKeepr - - - - just install?
Is there a list of destructions someplace?
With grateful thanks!!!
… Message ID: ***@***.***>
|
Beware, I recall some drama regarding their terms of service changes (e.g. bounty expiry) and general behaviour. I suggest careful research, if this is still a candidate solution. |
I installed your new version on windows, nice job! Patiently waiting for the DB conversion scripts. |
|
Im glat to found this perfect open project! I very hope it will be alive, and ready to donate. What i very miss - this is sort by parameter and display parameters in table. (ie uF, V, and etc). I know that it can be done by write searching rule, but it is not convenient. It will be great if you add feature to define columns per folder and display and sort by component paramets. Sory, im not programmer, but i can participate in test, suggest some features, and a bit donate. Im electorinc engineeer with a lot of components, and there is the first free and usefull solution for me and other hobbyst! |
|
The Limas rewrite sounds great, thank you for your efforts. Maybe that is interesting for some other guys here too, until the PartKeepr rewrite is finished. |
|
I'm also using PartDB, the interface and usability is not as good as Partkeepr but for now it's a good alternative. I've tried to import my Partkeepr database with the provided instructions but it just ended with a broken software, so i re - created everything. |
|
I switched to InvenTree a few years back, there is also a community data migration script from PartKeepr. |
|
@matmair tnx for introducing me to InvenTree! This looks already similar in terms of usability than partkeepr. |
|
I also switched to InvenTree and haven't looked back! Some things are a little clunky. E.g. clicking on a category goes to a page that only shows subcategories, and there's an extra click to get the actual parts in the category. There's also very limited support for batch editing. But overall it does the job and is under active development! |
|
@ikcalB I switched at least 2 years ago and I am involved in development now. I added everything I was missing 😆. |
This looks really neat. Are you aware of where the docs might be for pk2inventree? The getting started link on the repo is 404. |
|
@matmair sounds like a good idea! |
|
Limas got huge tech update: Symfony 7.0, Api-platform 3.2 ... and already has PK import script/cmd, simple install instructions some time i'm already testing docker-compose |
and others
Hello,
First I would like to thank Felicia.
She did a wonderful job for 10 years building this tool.
[Partkeepr as of 1.4.0]
As we can all see, this project is dead and stuck in Symphony 2.
The docker image is fully working but as some users pointed out, this leads to security issues and any feature request can be easily implemented. Octopart support is now also dead as Altium's API is very expensive.
Some users tried to move to Symphony 3 (which is outdated too) without any relevant success as a lot of things must be rewritten. This rewritten things would of course need to be rewritten again while updating to a newer version of Symphony.
I guess this project is stuck because of the legacy technology : as the 2 years not merged PR shows, writing old PHP / Symphony code is a rare and niche talent that few of us have.
[Why not re-writing ?]
Back in time when this project started, server side JavaScript didn't exist.
Now building a web based app with database management schema like Partkeepr is a lot easier with Node.js.
If some of you are interested (and have some knowledge in Javascript), maybe it's time to start a full rewriting with a newer, easier and maintained technology.
The text was updated successfully, but these errors were encountered: