ValueError: 5635 is not a valid FieldType

[MarceloCavalli]

Hi,

I'm tryng use the lib to read a database DBSIAM with this code.

from pydbisam import PyDBISAM

with PyDBISAM("/PATH_TO_DAT/FILE.dat") as db:
print(", ".join(db.fields()))
for row in db.rows():
print(", ".join(map(str, row)))

However, the code return this error:

ValueError: 5635 is not a valid FieldType

Can you help me?

[MarceloCavalli]

hi,

using the solution of FelipeSantos75, adding "MEMO = (5635, 1)" as a fieldtype i can read the file. The error its solved.

But, the return from this field memo is always "Fail".

This is the struture from table:

Table: D:\path to .dat
MD5 Checksum: 0x8ABE8E592364CB403D71D2E3BC64D001
Last Updated: 2025-05-02 11:11:34.858003
Total Fields: 33
Row Size: 512
Total Rows: 27637
Deleted Rows: 0
Description: None
User Version: 6.0

Column Info Offset: 512
Data Offset: 25856
Calc Row Area Size: 14150144
File Length: 14176000

Index, Name, Typ, Size, Row Offset
1, AUTO_INC, FieldType.STRING, 8, 26
2, LCTO, FieldType.STRING, 5, 35
3, QTDADE, FieldType.STRING, 7, 41
4, DESC_LANC, FieldType.STRING, 76, 49
5, OBS_LANC, FieldType.MEMO, 1, 126
6, OPERACAO, FieldType.STRING, 2, 135
7, DATA, FieldType.DATE, 4, 138

The dump to csv returns this
0398359, 0134, 1, "some description", Fail, E, 2025-04-01

Any ideas?

[linville]

The "Fail" is because pydbisam doesn't know how to decode the field.

In the Field::decode_from_row(..) method there are different decodings for each field type. The #6 PR is missing the entry to decode the MEMO field. Add a elif to handle MEMO, see if treating it like STRING gives any better output.

I haven't run across dbisam database that uses this field so there is not much further troubleshooting I can offer.

[X-Coder]

That won't work as easy as treating it like a string.

Data of Memo fields are BLOBS stored in a external file (table name + blb extension). It seems similar structured like the main table file, it seems to have the same checksum as the main table file.

From the official Memo data type description:

• it is variable length
• can contain up to 2 GB of data minus a NULL terminator
• the data stored is always text

There is also a Blob data type which is similar stored, but can contain binary data and not just text like the memo type.

I figured there is a definition of "blob block size" in the table header, by default it is 512 in the table editor and accepts values between 64-65536. This defines the size of each block in the blob file. It is at offset 0x33-0x35 (little endian), and can be read like this:

self._blob_block_size = int.from_bytes(self._data[0x33 : 0x33 + 3], byteorder="little")

I just don't know yet how the data is mapped to the row and where the number of blocks/size for the entry is stored.

The main question is, how would you dump binary, in case of blob data type, or with memo data type text (which could contain line breaks) in the csv, and up to 2 GB for each row's data field?

[maxthoursie]

I've implemented a basic support for MEMO and BLOB in #8

I've used the block size @X-Coder found but my examples where all using 512.

I choose to include MEMO in the csv by quoting and escaping strings. For BLOB, I write those to disk and reference the file using a content hash in the csv.

[X-Coder]

Thank you for your work. I think it is a good solution, inline for memo text and external file for binary blob data.
I added some info about 0xA in commit comment, which is the row index.

@linville, the checksum 0x9 in all files is not a file checksum, it is a engine signature: only engines matching this signature are able to access the files.

[MarceloCavalli]

Hi, i tested your solution and it's work!

Thank you so much.

Perhapes be interessting update the driver on pip's repository.

I've implemented a basic support for MEMO and BLOB in #8

I've used the block size @X-Coder found but my examples where all using 512.

I choose to include MEMO in the csv by quoting and escaping strings. For BLOB, I write those to disk and reference the file using a content hash in the csv.