Autor Thema: DLOG_STORE_FILE_CSV append  (Gelesen 1073 mal)

0 Mitglieder und 1 Gast betrachten dieses Thema.

Offline laruso

  • Newbie
  • *
  • Beiträge: 3
    • Profil anzeigen
    • E-Mail
DLOG_STORE_FILE_CSV append
« am: 03. Mai 2021, 14:56:30 »
First, thanks so much for distributing the OSCAT libraries!

I want to use the function block "DLOG_STORE_FILE_CSV" to log data into one file only. Because of the restrictions of retaining variables (for BeagleBone Black the proper shutdown is needed), I want to do some workaround in that way as the data has to be append only to the file.
I know this feature already exists, but to work properly the data in the variable "save_data : DLOG_SAVE" (used as a paramter for "DLOG_STORE_FILE_CSV" instance, has to be a retained variable. Therefore it is necessary to use a UPS, to savely retain variable on my BeagleBone Black. To workaround this, I would like to use an "append" parameter, to not need the variable to be retained.

If I use the code and the demo from the library (on my BeagleBone Black without retaining the retain variables), the content of the file will be erased, due to the empty variable "save_data" of data type "DLOG_SAVE" used as parameter for instance call of "DLOG_STORE_FILE_CSV". For me the functionality of writing with append, should not depend on retaining any variables.

(Btw. I know that that hasn't already be written to file will be lost due to power lost; but this doesn't matter. Morte important is the fact that there is no unlimited storage on devices, so it would be best for me to define a maximum number of "FILE_SIZE" or "line count", to delete the latest log entry and write the new log entry, if the file hits this maximum file size. So there will be only one file with the latest log entries, depending on the defined file size.)

At least the question is how to change the behavior of writing to file (append or delete content and write data) if there is already a file with that filename?

I've read a lot about this function block in this forum. Several times there is the hint to use the append mode, but don't know how to do. Can anyone please give me an advise in how to code that?

In the documentation for the function block "FILE_SERVER" there is also a hint:
Zitat
An automatic append function can be realized very easy. The parameter FILE_SIZE has to be written to the OFFSET
But I don't know how to implement. When I tried to do it outside the function block instance call, an error occurs because FILE_SIZE is no input of FILE_SERVER. Here is the code I tried:

x1(X:=x); // Logger input for STRING, used for Time stamp
x2(X:=x, STR:=Glob_Var.sLogExportText2); // Logger input for STRING, used for Event type
x3(X:=x, STR:=Glob_Var.sLogExportText3); // Logger input for STRING, used for Description
DLOG_STORE_FILE_CSV(X:=x, SAVE_DATA:=save_data, ENABLE:=enable, TRIG_M:=Glob_Var.xLogExportTrig_m, FILENAME:=filename, DTI:=RTC_2.UDT, SEP:=9, AUTO_CLOSE:=TIME#10S, ERROR_C=>error_c, ERROR_T=>error_t); // File storage (csv)

RTrig(CLK:=DLOG_STORE_FILE_TSV.FSD.FILE_OPEN);

IF RTrig.Q THEN
DLOG_STORE_FILE_CSV.FSD.OFFSET := DLOG_STORE_FILE_CSV.FSD.FILE_SIZE;
END_IF

When I try to use a copy of the function block "DLOG_STORE_FILE_CSV" from the library everything is OK, but when I try to use a copy of the function block "FILE_SERVER" from the library and to the change there, it gets me several errors (32) when I want to translate the project. Here is the code I tried:

150: (* File Open *)
IF command = BYTE#0 THEN
IF error THEN
step := 30000; (* Beenden *)
ELSE
(* Filedaten eintragen *)
handle := open_handle;
used_filename := FSD.FILENAME;
file_position := UDINT#4294967295; (* maximal = unbekannt *)
FSD.FILE_OPEN := TRUE;
FSD.OFFSET := FSD.FILE_SIZE;
step := 200;
END_IF;
END_IF;


Till now my workaround was to change the code of "DLOG_STORE_FILE_CSV" like this:
10: X.UCB.D_STRING := fn_last;
// IF fn_last = SAVE_DATA.FN_REM THEN
X.UCB.D_HEAD := WORD#16#F201; (* Filemode open + write *)
X.ADD_COM := 00;
// ELSE
// X.UCB.D_HEAD := WORD#16#F301; (* Filemode create + write *)
// X.ADD_COM := 02; (* ADD HEADER *)
// END_IF;

X.UCB.D_MODE := 1; (* Dateinamen als String ablegen *)
UCB(DATA:=X.UCB); (* Daten eintragen *)
step_1 := 30;
But this results in losing the opertunity to get a new file, by changing the variable "filename", and this is no goal.


I use CODESYS 3.5.16 Patch 4 and the OSCAT NETWORK library (v1.3.5.2) from CODESYS store.

I'm over this topic for a long while now and pretty looking forward for some help to get it work.
« Letzte Änderung: 03. Mai 2021, 15:38:37 von laruso »

Offline peewit

  • Moderator
  • *****
  • Beiträge: 2 366
    • Profil anzeigen
Re: DLOG_STORE_FILE_CSV append
« Antwort #1 am: 18. Mai 2021, 22:26:33 »
the problem with the limited storage space can be avoided by using several different but always recurring file names.
this way after a certain amount of files the old ones will be overwritten again and again

documentation dlog_store_csv module

With parameters FILENAME the fle name (including path if necessary) is
defned. If the flename is changed during the recprding, it will
automatically on-the-fy changed to the new record fle (with no data loss).
This change can also be automated. The parameter FILE NAME supports
the use of date / time parameter (see documentation from the module
DT_TO_STRF)
Example: FILE NAME = 'Station_01_#R.csv'
At position of '#R' automatically the current minute number is entered. This means
that automatically every minute the file name changes, and therefore the data is
written into the file. Thus, within an entire hour 60 files are created and filled with
data, and in the ring buffer manner overwritten again and again.
53 Version 1.21
Chapter 7. Data Logger
A recording can be done automatically and creates every day, week, month, etc. a
new file as desired. If a new FILE NAME is detected, a possibly existing file is erased
and rewritten.

Offline laruso

  • Newbie
  • *
  • Beiträge: 3
    • Profil anzeigen
    • E-Mail
Re: DLOG_STORE_FILE_CSV append
« Antwort #2 am: 02. Juni 2021, 15:04:42 »
I really think you didn't get my issue, because I don't have problems with limited storage space. Albeit a limited storage problem can't be fixed with saving data in several files instead of one.

My issue is, that I only want to append the data if the file already exists, without the need of the variable "save_data : DLOG_SAVE" to be retained. At the end there are two cases:

- The file exist: Append the data
- The file doesn't exist: Create the file and write the data

I think this will be in general the better solution. Instead of doing so with retained variables, it would have the oportunity to store data to a file that wasn't created by the function DLOG_STORE_FILE_CSV itself. A switch to append or not to append in an alreadey existing file would be best of all.

My use case is to log process data and to have a look into it, if it's necessary. No chance to delete the file(s) during runtime. Therefore a style of shifting register, with an configurable absolute size/entry limit is needed. If for example 1000 lines are filled in the file, the oldest line has to be deleted and the new entry will be appended.

Can anybody at least help me changing the code, please?

PS: Gerne kann ich mich auch in Deutsch ausdrücken und dachte es würde wohl in Englisch eine Mehrzahl an Leute erreichen.
« Letzte Änderung: 02. Juni 2021, 15:30:27 von laruso »

Offline peewit

  • Moderator
  • *****
  • Beiträge: 2 366
    • Profil anzeigen
Re: DLOG_STORE_FILE_CSV append
« Antwort #3 am: 04. Juni 2021, 16:14:41 »
Therefore a kind of shift register is needed, with a configurable absolute size/input limit. If e.g. 1000 lines are filled in the file, the oldest line must be deleted and the new entry is appended.


this is not easily feasible and is a completely different concept
you can't just delete a few lines from a file
for this you have to load the complete file into a bytearray in the sps
then pick out the lines and move the rest of the data and save it again.

it would be easier if you use the "LOG_CONTROL" block right away
it has a ringbuffer - there happens what you want automatically

Translated with www.DeepL.com/Translator (free version)