Commit Graph

30 Commits

Author SHA1 Message Date
gezimbll
3ebf3007a5 added startdelay config option for ers readers 2025-01-29 14:44:28 +01:00
ionutboangiu
63c1358c3f Update behaviour of file readers using inotify
Ensure that files already existing in the source path are processed
before the reader starts listening for filesystem change events.
2024-09-23 20:28:26 +02:00
ionutboangiu
65e8128303 Implement ErSv1.RunReader api 2024-09-23 20:28:26 +02:00
ionutboangiu
e1adb674b8 Remove redundant dir parameter from processFile methods
Applies to both file readers and loader (for loader, the blank statement
was used anyway).

It's redundant because for file readers, the rdr.sourceDir value was
always passed as the parameter when it was already part of the method's
object.

Parameter had to also be removed from the WatchDir function and the
functions it depends on.
2024-09-23 20:28:26 +02:00
ionutboangiu
7998ed6a0f Extract common dir processing logic to a func
It will be reused for all file readers.
Rename rdrDir field to sourceDir.
2024-09-23 20:28:26 +02:00
arberkatellari
c7221f0d1e Add reader id within the *vars in ERs 2024-08-12 18:34:50 +02:00
ionutboangiu
113e2a2bdf Replace interface{} with any 2023-05-31 10:22:28 +02:00
adi
5d23b6433f Chanegd FileName consts into ers 2022-11-20 18:46:55 +01:00
andronache98
ecdf5e54ea Changed headers 2022-03-09 14:14:30 +01:00
andronache98
9ec56f5155 Changed headers 2022-03-09 14:14:30 +01:00
Trial97
fc1c7e8c79 Updated tests 2021-05-26 15:54:36 +02:00
Trial97
c28ac62968 Updated flatstore tests 2021-05-19 15:54:14 +02:00
Trial97
d868c0a5f0 Added partial reader 2021-05-19 15:54:14 +02:00
Trial97
f6863bad3e Updated flatsore ers 2021-05-06 14:00:37 +02:00
Trial97
8aa7009d36 Updated AttributesSv1 APIs 2021-04-13 13:35:03 +02:00
Trial97
2da495af19 Starting adding back AttributesSv1 APIs 2021-04-13 13:35:03 +02:00
nickolasdaniel
62f435bdae Removing unused variables in utils/consts.go 2021-04-02 13:42:52 +02:00
Trial97
c13a5cf790 Finished implementing the DataNode as a replacement 2021-03-26 13:46:25 +01:00
Trial97
ba090352ec Updated agents tests 2021-03-26 13:46:25 +01:00
Trial97
e52e8d7622 Added Length Field in case of NMSlice and support for multiple indexes in template path 2021-03-18 16:46:55 +01:00
Trial97
5353b95e43 Updated go version and replaced ioutil package 2021-03-18 16:46:55 +01:00
TeoV
2bfb234804 Update integration test in agents 2021-01-15 09:53:55 +01:00
TeoV
906baa051a Build success 2021-01-14 10:30:35 +01:00
Trial97
06e4fd13c3 Updated integration tests 2020-12-09 17:00:04 +01:00
Trial97
505f230b1e Added inotify support for loaders 2020-11-11 15:06:35 +01:00
TeoV
24b2206a2a Return error in case of filterS.Pass 2020-05-20 17:20:34 +02:00
Trial97
f10f570a27 Removed config.NavigableMap 2020-05-18 13:26:06 +02:00
Trial97
13e813f848 Fixed agents unit tests 2020-05-07 11:17:41 +02:00
Trial97
41e35bc571 Added Opts to APIs 2020-04-26 13:08:34 +02:00
TeoV
eaa3469aa2 Add test for *json type for ERs 2020-04-01 16:27:55 +02:00