Remove old entries in harvest_job_table

[frafra]

clean-harvest-log removes old logs, but I have not found anything similar to clean harvest_job_table. Having automatic harvesting means ending up with a forever-growing, non paginated/non-searchable job page, which seems pretty useless to me.

What about keeping the last 100 jobs by default and/or having a clean-up procedure? Like clean-harvest-jobs.

[seitenbau-govdata]

I think there is already a command clear-history

ckanext-harvest/ckanext/harvest/cli.py

Line 112 in d84d847

def clear_history(ctx, id):

or rather clearsource_history

ckanext-harvest/ckanext/harvest/commands/harvester.py

Line 220 in d84d847

elif cmd == "clearsource_history":

for deleting the old harvest jobs and the related objects. But at the moment the command still deletes all old and current harvest jobs and harvest objects. But we already working on a pull request with an updated version which keeps at least the running harvest jobs and latest harvest objects.

[seitenbau-govdata]

The pull request is now available. #484