Commit Graph

38 Commits

Author SHA1 Message Date
ionutboangiu
e05be683ae change license from GPLv3 to AGPLv3 2025-10-13 09:57:41 +02:00
gezimbll
3ebf3007a5 added startdelay config option for ers readers 2025-01-29 14:44:28 +01:00
ionutboangiu
d8acee74e2 Add *fileLineNumber var for file readers 2024-11-03 13:19:45 +01:00
ionutboangiu
63c1358c3f Update behaviour of file readers using inotify
Ensure that files already existing in the source path are processed
before the reader starts listening for filesystem change events.
2024-09-23 20:28:26 +02:00
ionutboangiu
65e8128303 Implement ErSv1.RunReader api 2024-09-23 20:28:26 +02:00
ionutboangiu
e1adb674b8 Remove redundant dir parameter from processFile methods
Applies to both file readers and loader (for loader, the blank statement
was used anyway).

It's redundant because for file readers, the rdr.sourceDir value was
always passed as the parameter when it was already part of the method's
object.

Parameter had to also be removed from the WatchDir function and the
functions it depends on.
2024-09-23 20:28:26 +02:00
ionutboangiu
7998ed6a0f Extract common dir processing logic to a func
It will be reused for all file readers.
Rename rdrDir field to sourceDir.
2024-09-23 20:28:26 +02:00
arberkatellari
c7221f0d1e Add reader id within the *vars in ERs 2024-08-12 18:34:50 +02:00
ionutboangiu
4c34d321de Add sanity check to prevent xml reader panic
HierarchyPath parser now returns nil when
the path is empty (instead of a string slice with one
EmptyString element).

If isAbsolute is set to true, when calling the AsString
method on a nil HierarchyPath, only the separator will
be returned. Alternatively, if isAbsolute is false, it
will just return . to signal that the path currently
being processed is the one required.This avoids a nil
expr error coming from the xmlquery library.

Use the Query and QueryAll functions from the xmlquery
package to be able to handle the errors ourselves and
avoid panics.

Remove config default value for xmlRootPath. The field
will remain commented in config_defaults for reference.

Add tests for HierarchyPath.AsString function.

Add comments for XmlProvider and xmlRootPath opt.

Update XmlProvider String() func based on the comment
which specifies that it should display the already
parsed values out of cache.
2023-10-19 17:19:30 +02:00
adi
5d23b6433f Chanegd FileName consts into ers 2022-11-20 18:46:55 +01:00
andronache98
ecdf5e54ea Changed headers 2022-03-09 14:14:30 +01:00
andronache98
9ec56f5155 Changed headers 2022-03-09 14:14:30 +01:00
ionutboangiu
aa46b1eefd Fix compilation errors after making ers opts into structs 2021-11-01 13:51:02 +02:00
Trial97
c28ac62968 Updated flatstore tests 2021-05-19 15:54:14 +02:00
Trial97
d868c0a5f0 Added partial reader 2021-05-19 15:54:14 +02:00
Trial97
f6863bad3e Updated flatsore ers 2021-05-06 14:00:37 +02:00
Trial97
7aeeeec50c Updated ers config 2021-05-04 20:45:09 +02:00
Trial97
2da495af19 Starting adding back AttributesSv1 APIs 2021-04-13 13:35:03 +02:00
Trial97
c13a5cf790 Finished implementing the DataNode as a replacement 2021-03-26 13:46:25 +01:00
Trial97
ba090352ec Updated agents tests 2021-03-26 13:46:25 +01:00
Trial97
e52e8d7622 Added Length Field in case of NMSlice and support for multiple indexes in template path 2021-03-18 16:46:55 +01:00
Trial97
5353b95e43 Updated go version and replaced ioutil package 2021-03-18 16:46:55 +01:00
TeoV
2bfb234804 Update integration test in agents 2021-01-15 09:53:55 +01:00
TeoV
906baa051a Build success 2021-01-14 10:30:35 +01:00
Trial97
06e4fd13c3 Updated integration tests 2020-12-09 17:00:04 +01:00
Trial97
c539800acd Added more Clone functions for config sections 2020-11-27 08:41:56 +01:00
Trial97
2b19127d05 Added more Clone functions for config sections 2020-11-27 08:41:56 +01:00
Trial97
505f230b1e Added inotify support for loaders 2020-11-11 15:06:35 +01:00
TeoV
24b2206a2a Return error in case of filterS.Pass 2020-05-20 17:20:34 +02:00
Trial97
13e813f848 Fixed agents unit tests 2020-05-07 11:17:41 +02:00
Trial97
41e35bc571 Added Opts to APIs 2020-04-26 13:08:34 +02:00
Trial97
f4e9d39558 Added the file name in *vars for csv,fwv,xml,flatstore and partial_csv event readers. Closes #407 2020-03-24 16:37:59 +01:00
TeoV
3e3fc6c457 Correctly populate ConcurrentRequest from config in EventReader fixes #1932 2020-02-18 14:24:50 +01:00
TeoV
b85ebe8214 Update EventReader integration tests 2020-02-05 15:57:58 +01:00
TeoV
52f0e0a86c Merge ContentFields,HeaderFields,TrailerFields into one Files 2020-01-29 14:40:57 +01:00
TeoV
26ea469328 Merge content,header,trailer fields into fields 2020-01-29 14:40:57 +01:00
TeoV
f74df27ddd Remove pathPrfx from xmlDP 2020-01-23 10:56:30 +02:00
TeoV
4ea9cd3a48 Add XML to EventReader and test for it 2020-01-11 14:52:37 +02:00