Skip to content

Commit 54c4613

Browse files
BillFarberBillFarber
authored
Merge pull request #68 from marklogic/develop
Release 1.2.0
2 parents be1e6e6 + 4185178 commit 54c4613

21 files changed

+391
-29
lines changed

‎.vscode/settings.json

+9-8
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,14 @@
11
{
2-
"python.formatting.provider": "black",
3-
"python.linting.enabled": true,
4-
"python.linting.flake8Enabled": true,
5-
"python.linting.flake8Args": [
6-
"--max-line-length=88"
7-
],
2+
"[python]": {
3+
"editor.defaultFormatter": "ms-python.black-formatter",
4+
"editor.formatOnSave": true
5+
},
6+
"python.testing.unittestEnabled": false,
7+
"python.testing.pytestEnabled": true,
88
"python.testing.pytestArgs": [
99
"tests"
1010
],
11-
"python.testing.unittestEnabled": false,
12-
"python.testing.pytestEnabled": true,
11+
"flake8.args": [
12+
"--max-line-length=120"
13+
]
1314
}

‎CONTRIBUTING.md

+22-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,16 @@ To try this out locally:
66
- `source .venv/bin/activate` to use that virtual environment.
77
- `poetry install` to install project dependencies.
88

9-
VSCode is recommended for development. You can try [these instructions](https://www.pythoncheatsheet.org/blog/python-projects-with-poetry-and-vscode-part-1)
9+
VSCode is recommended for development.
10+
- For formatting, the project uses the [Black](https://github.com/psf/black) code formatter, and the
11+
[Black Formatter VSCode extension](https://marketplace.visualstudio.com/items?itemName=ms-python.black-formatter)
12+
is recommended.
13+
- For linting, the project uses the [Flake8](https://flake8.pycqa.org/en/latest/) linter and the
14+
[Flake8 extension](https://marketplace.visualstudio.com/items?itemName=ms-python.flake8) is recommended.
15+
These tools are included in the project by pyproject.toml and the settings are in .vscode/settings.json.
16+
17+
You can also get additional information at
18+
[these instructions](https://www.pythoncheatsheet.org/blog/python-projects-with-poetry-and-vscode-part-1)
1019
for getting setup in VSCode with linting and formatting enabled.
1120

1221
## Running the tests
@@ -67,6 +76,18 @@ project's documentation:
6776

6877
python -i shell/docs.py
6978

79+
80+
## Testing updates in a different local project
81+
If you are using this in another project and making changes for it, use the following command to make the
82+
changes to this local project immediately reflected in a dependent project:
83+
```poetry add <local-path-to-this-project>/marklogic-python-client/```
84+
85+
Using this method will allow you to very easily test changes to this project, in a different local project.
86+
87+
Keep in mind that you probably do not want to check that version of the pyproject.toml file into version
88+
control since it is only useful locally.
89+
90+
7091
## Testing the documentation locally
7192

7293
The docs for this project are stored in the `./docs` directory as a set of Markdown files. These are published via

‎Jenkinsfile

+39
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
@Library('shared-libraries') _
2+
pipeline{
3+
agent none;
4+
environment{
5+
JAVA_HOME_DIR="/home/builder/java/jdk-11.0.2"
6+
GRADLE_DIR =".gradle"
7+
}
8+
options {
9+
checkoutToSubdirectory 'marklogic-python-client'
10+
buildDiscarder logRotator(artifactDaysToKeepStr: '7', artifactNumToKeepStr: '', daysToKeepStr: '30', numToKeepStr: '')
11+
}
12+
stages{
13+
stage('tests'){
14+
agent {label 'devExpLinuxPool'}
15+
steps{
16+
script{
17+
copyRPM 'Latest','11'
18+
setUpML '$WORKSPACE/xdmp/src/Mark*.rpm'
19+
sh label:'deploy project', script: '''#!/bin/bash
20+
export JAVA_HOME=$JAVA_HOME_DIR
21+
export GRADLE_USER_HOME=$WORKSPACE/$GRADLE_DIR
22+
export PATH=$GRADLE_USER_HOME:$JAVA_HOME/bin:$PATH
23+
cd marklogic-python-client/test-app
24+
./gradlew -i mlDeploy -PmlPassword=admin
25+
'''
26+
sh label:'Run tests', script: '''#!/bin/bash
27+
cd marklogic-python-client
28+
python -m venv .venv;
29+
source .venv/bin/activate;
30+
pip install poetry;
31+
poetry install;
32+
pytest --junitxml=TestReport.xml || true
33+
'''
34+
junit 'marklogic-python-client/TestReport.xml'
35+
}
36+
}
37+
}
38+
}
39+
}

‎docs/creating-client.md

+6
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,12 @@ The `Client` class is the primary API to interact with in the MarkLogic Python c
1111
found in both the `Session` class and the `requests` API. You can therefore use a `Client` object in the same manner
1212
as you'd use either the `Session` class or the `requests` API.
1313

14+
## Table of contents
15+
{: .no_toc .text-delta }
16+
17+
- TOC
18+
{:toc}
19+
1420
## Creating a client
1521

1622
A `Client` instance can be created either by providing a base URL for all requests along with authentication:

‎docs/eval.md

+20
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ execution of custom code, whether via an inline script or an existing module in
99
The MarkLogic Python client supports execution of custom code by simplifying the submission of custom code
1010
and converting the multipart response into more useful Python data types.
1111

12+
## Table of contents
13+
{: .no_toc .text-delta }
14+
15+
- TOC
16+
{:toc}
17+
1218
## Setup
1319

1420
The examples below all depend on the instructions in the [setup guide](example-setup.md) having already been performed.
@@ -117,3 +123,17 @@ processing of the response or debugging requests.
117123
The `client.eval` and `client.invoke` functions both support referencing a
118124
[REST API transaction](https://docs.marklogic.com/REST/client/transaction-management) via the `tx`
119125
argument. See [the guide on transactions](transactions.md) for further information.
126+
127+
## Providing additional arguments
128+
129+
The `client.eval` and `client.invoke` methods each provide a `**kwargs` argument, so you can pass in any other arguments you would
130+
normally pass to `requests`. For example:
131+
132+
```
133+
client.eval(javascript="fn.currentDateTime()", params={"database": "Documents"})
134+
client.invoke("/sample.sjs", params={"database": "Documents"})
135+
```
136+
137+
Please see [the eval endpoint documentation](https://docs.marklogic.com/REST/POST/v1/eval)
138+
and [the invoke endpoint documentation](https://docs.marklogic.com/REST/POST/v1/invoke) for
139+
information on additional parameters.

‎docs/managing-documents/reading.md

+6
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,12 @@ The [GET /v1/documents](https://docs.marklogic.com/REST/GET/v1/documents) endpoi
1010
reading multiple documents with metadata via a multipart/mixed HTTP response. The MarkLogic Python client simplifies
1111
handling the response by converting it into a list of `Document` instances via the `client.documents.read` method.
1212

13+
## Table of contents
14+
{: .no_toc .text-delta }
15+
16+
- TOC
17+
{:toc}
18+
1319
## Setup for examples
1420

1521
The examples below all assume that you have created a new MarkLogic user named "python-user" as described in the

‎docs/managing-documents/searching.md

+6
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,12 @@ returning content and metadata for each matching document. Similar to reading mu
1212
HTTP response. The MarkLogic Python client simplifies use of this operation by returning a list of `Document` instances
1313
via the `client.documents.search` method.
1414

15+
## Table of contents
16+
{: .no_toc .text-delta }
17+
18+
- TOC
19+
{:toc}
20+
1521
## Setup for examples
1622

1723
The examples below all assume that you have created a new MarkLogic user named "python-user" as described in the

‎docs/managing-documents/writing.md

+8
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,14 @@ writing multiple documents with metadata via a multipart HTTP request. The MarkL
1111
simplifies the use of this endpoint via the `client.documents.write` method and the `Document`
1212
class.
1313

14+
## Table of contents
15+
{: .no_toc .text-delta }
16+
17+
- TOC
18+
{:toc}
19+
20+
## Setup
21+
1422
The examples below all assume that you have created a new MarkLogic user named "python-user" as described in the
1523
[setup guide](../example-setup.md). In addition, each of the examples below requires the following `Client` instance to be created
1624
first:

‎docs/rows.md

+21-1
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,12 @@ The [MarkLogic REST rows service](https://docs.marklogic.com/REST/client/row-man
99
operations for querying for rows via several query languages. The MarkLogic Python client simplifies submitting queries
1010
for rows and converting responses into useful data structures.
1111

12+
## Table of contents
13+
{: .no_toc .text-delta }
14+
15+
- TOC
16+
{:toc}
17+
1218
## Setup
1319

1420
The examples below require documents to be loaded along with a
@@ -178,4 +184,18 @@ Printing the `df` object will yield the following:
178184
1 Davis Miles 1926-05-26
179185
2 Armstrong Louis 1901-08-04
180186
3 Coltrane John 1926-09-23
181-
```
187+
```
188+
189+
## Providing additional arguments
190+
191+
The `client.rows.query` method provides a `**kwargs` argument, so you can pass in any other arguments you would
192+
normally pass to `requests`. For example:
193+
194+
```
195+
response = client.rows.query("op.fromView('example', 'musician')", params={"database": "Documents"})
196+
```
197+
198+
Please see [the rows endpoint documentation](https://docs.marklogic.com/REST/POST/v1/rows) for
199+
information on additional parameters. If you are submitting a GraphQL query, then see
200+
[the GraphQL endpoint documentation](https://docs.marklogic.com/REST/POST/v1/rows/graphql) for
201+
information on parameters for that endpoint.

‎docs/transactions.md

+8
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,14 @@ via a `Transaction` class that is also a
1313
thereby allowing it to handle committing or rolling back the transaction without any user
1414
involvement.
1515

16+
## Table of contents
17+
{: .no_toc .text-delta }
18+
19+
- TOC
20+
{:toc}
21+
22+
## Using a transaction
23+
1624
The following example demonstrates writing documents via multiple calls to MarkLogic,
1725
all within the same REST API transaction; the example depends on first following the
1826
instructions in the [setup guide](example-setup.md):

‎marklogic/documents.py

+20-13
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import json
22
from collections import OrderedDict
3+
from email.message import Message
34
from typing import Union
45

56
from marklogic.transactions import Transaction
@@ -262,23 +263,29 @@ def _extract_values_from_header(part) -> dict:
262263
Returns a dict containing values about the document content or metadata.
263264
"""
264265
encoding = part.encoding
265-
disposition = part.headers["Content-Disposition".encode(encoding)].decode(encoding)
266-
disposition_values = {}
267-
for item in disposition.split(";"):
268-
tokens = item.split("=")
269-
# The first item will be "attachment" and can be ignored.
270-
if len(tokens) == 2:
271-
disposition_values[tokens[0].strip()] = tokens[1]
266+
disposition = part.headers["Content-Disposition".encode(encoding)].decode(
267+
encoding
268+
)
272269

273270
content_type = None
274271
if part.headers.get("Content-Type".encode(encoding)):
275-
content_type = part.headers["Content-Type".encode(encoding)].decode(encoding)
272+
content_type = part.headers["Content-Type".encode(encoding)].decode(
273+
encoding
274+
)
276275

277-
uri = disposition_values["filename"]
278-
if uri.startswith('"'):
279-
uri = uri[1:]
280-
if uri.endswith('"'):
281-
uri = uri[:-1]
276+
content_disposition_header = part.headers[
277+
"Content-Disposition".encode(encoding)
278+
].decode(encoding)
279+
msg = Message()
280+
msg["content-disposition"] = content_disposition_header
281+
uri = msg.get_filename()
282+
283+
disposition_values = {}
284+
for item in disposition.replace(uri, "").split(";"):
285+
tokens = item.split("=")
286+
key = tokens[0].strip()
287+
if key in ["category", "versionId"]:
288+
disposition_values[key] = tokens[1]
282289

283290
return {
284291
"uri": uri,

‎marklogic/rows.py

+57
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,63 @@ def query(
6767
not 2xx, then the entire response is always returned.
6868
"""
6969
path = "v1/rows/graphql" if graphql else "v1/rows"
70+
return self.__send_request(
71+
path,
72+
dsl,
73+
plan,
74+
sql,
75+
sparql,
76+
graphql,
77+
format,
78+
tx,
79+
return_response,
80+
**kwargs,
81+
)
82+
83+
def update(
84+
self,
85+
dsl: str = None,
86+
plan: dict = None,
87+
format: str = "json",
88+
tx: Transaction = None,
89+
return_response: bool = False,
90+
**kwargs,
91+
):
92+
"""
93+
Sends an update query to an endpoint at the MarkLogic rows service defined at
94+
https://docs.marklogic.com/REST/client/row-management. One of 'dsl' or
95+
'plan' must be defined. This feature requires the use of MarkLogic version
96+
11.2 or later.
97+
98+
For more information about Optic Update and using the Optic DSL,
99+
see https://docs.marklogic.com/guide/app-dev/OpticAPI.
100+
101+
:param dsl: an Optic DSL query
102+
:param plan: a serialized Optic query
103+
:param tx: optional REST transaction in which to service this request.
104+
:param return_response: boolean specifying if the entire original response
105+
object should be returned (True) or if only the data should be returned (False)
106+
upon a success (2xx) response. Note that if the status code of the response is
107+
not 2xx, then the entire response is always returned.
108+
"""
109+
path = "v1/rows/update"
110+
return self.__send_request(
111+
path, dsl, plan, None, None, None, format, tx, return_response, **kwargs
112+
)
113+
114+
def __send_request(
115+
self,
116+
path: str = None,
117+
dsl: str = None,
118+
plan: dict = None,
119+
sql: str = None,
120+
sparql: str = None,
121+
graphql: str = None,
122+
format: str = "json",
123+
tx: Transaction = None,
124+
return_response: bool = False,
125+
**kwargs,
126+
):
70127
headers = kwargs.pop("headers", {})
71128
data = None
72129
if graphql:

‎pyproject.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[tool.poetry]
22
name = "marklogic-python-client"
3-
version = "1.1.0"
3+
version = "1.2.0"
44
description = "Python client for MarkLogic, built on the requests library"
55
authors = ["MarkLogic <[email protected]>"]
66
readme = "README.md"

‎test-app/.gitignore

+1
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
.gradle
22
gradle-local.properties
33
build
4+
docker

‎test-app/docker-compose.yml

+18
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
version: '3.8'
2+
name: marklogic_python
3+
4+
services:
5+
6+
marklogic:
7+
image: "marklogicdb/marklogic-db:11.2.0-centos-1.1.2"
8+
platform: linux/amd64
9+
environment:
10+
- INSTALL_CONVERTERS=true
11+
- MARKLOGIC_INIT=true
12+
- MARKLOGIC_ADMIN_USERNAME=admin
13+
- MARKLOGIC_ADMIN_PASSWORD=admin
14+
volumes:
15+
- ./docker/marklogic/logs:/var/opt/MarkLogic/Logs
16+
ports:
17+
- "8000-8002:8000-8002"
18+
- "8030-8031:8030-8031"

‎test-app/src/main/ml-data/doc2;copy.xml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
<hello>semicolon</hello>

‎test-app/src/main/ml-data/doc2=copy.xml

+1
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
<hello>equal</hello>

0 commit comments

Comments
 (0)