State Management - Set initial State or update State through GUI or API

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

State Management - Set initial State or update State through GUI or API

Peter Wicks (pwicks)
I'm using the QueryDatabaseTable processor.  Frequently I would like to set an initial state value for my columns max value, or I'd like to change the value stored in State Management for that column, this comes up in operational scenarios ("reload all data after this exact timestamp, that's when everything went bad").

But in looking through the code, and even the rest API, I don't see any options to set/modify a processes state.  Am I missing something? Is there a way to set initial values like I want?

Regards,
  Peter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State Management - Set initial State or update State through GUI or API

Matt Burgess
I believe that was a design decision to ensure for clusters and such that state could not be modified on the fly (and thus possibly become inconsistent or alter what would be a normal provenance trace). Processors internally are responsible for maintaining the state they wish to keep, I don't think it's possible to tweak it in order to change the behavior of the processors.

Having said that, there are much more knowledgeable folks than me re: state management, hopefully they can clarify/correct my answer :)

Regards,
Matt

> On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <[hidden email]> wrote:
>
> I'm using the QueryDatabaseTable processor.  Frequently I would like to set an initial state value for my columns max value, or I'd like to change the value stored in State Management for that column, this comes up in operational scenarios ("reload all data after this exact timestamp, that's when everything went bad").
>
> But in looking through the code, and even the rest API, I don't see any options to set/modify a processes state.  Am I missing something? Is there a way to set initial values like I want?
>
> Regards,
>  Peter
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State Management - Set initial State or update State through GUI or API

Bryan Bende
We do allow users to view and clear the state for a given processor, which
would set you back at the beginning.

I'm wondering if there would be any downside to letting the users set the
value of one of the keys in the state map on the view state screen.
Seems like a cool feature to be able to start back at a specific value,
rather than have to start at the beginning, but it also opens up someone
entering something that causes bad behavior for the processor.


On Thu, Jul 28, 2016 at 3:55 PM, Matt Burgess <[hidden email]> wrote:

> I believe that was a design decision to ensure for clusters and such that
> state could not be modified on the fly (and thus possibly become
> inconsistent or alter what would be a normal provenance trace). Processors
> internally are responsible for maintaining the state they wish to keep, I
> don't think it's possible to tweak it in order to change the behavior of
> the processors.
>
> Having said that, there are much more knowledgeable folks than me re:
> state management, hopefully they can clarify/correct my answer :)
>
> Regards,
> Matt
>
> > On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <[hidden email]>
> wrote:
> >
> > I'm using the QueryDatabaseTable processor.  Frequently I would like to
> set an initial state value for my columns max value, or I'd like to change
> the value stored in State Management for that column, this comes up in
> operational scenarios ("reload all data after this exact timestamp, that's
> when everything went bad").
> >
> > But in looking through the code, and even the rest API, I don't see any
> options to set/modify a processes state.  Am I missing something? Is there
> a way to set initial values like I want?
> >
> > Regards,
> >  Peter
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State Management - Set initial State or update State through GUI or API

Joe Witt
Hello,

The idea is definitely a good one.  I'd propose that rather than
making state management editable (other than clearing/reseting it)
that a given processor should expose a property or properties to allow
a user to express their needs in terms of starting point.  If we were
to allow altering state then that too becomes an interface we have to
honor and it wouldn't offer much beyond the processor properties
themselves which have a well defined lifecycle, documentation,
interface, etc..  With that in mind I see state management as is to be
a valuable tool but really one that allows one to 'look behind the
wall' to see the internal state of a processor but not to interact
with it other than saying "reset" which can mean whatever reset might
mean for that processor.

So for this QueryDatabase processor perhaps a JIRA to add support for
a property or couple of properties that allows the original intent of
this thread to be captured.

Does that sound reasonable?

Thanks
Joe

On Thu, Jul 28, 2016 at 4:28 PM, Bryan Bende <[hidden email]> wrote:

> We do allow users to view and clear the state for a given processor, which
> would set you back at the beginning.
>
> I'm wondering if there would be any downside to letting the users set the
> value of one of the keys in the state map on the view state screen.
> Seems like a cool feature to be able to start back at a specific value,
> rather than have to start at the beginning, but it also opens up someone
> entering something that causes bad behavior for the processor.
>
>
> On Thu, Jul 28, 2016 at 3:55 PM, Matt Burgess <[hidden email]> wrote:
>
>> I believe that was a design decision to ensure for clusters and such that
>> state could not be modified on the fly (and thus possibly become
>> inconsistent or alter what would be a normal provenance trace). Processors
>> internally are responsible for maintaining the state they wish to keep, I
>> don't think it's possible to tweak it in order to change the behavior of
>> the processors.
>>
>> Having said that, there are much more knowledgeable folks than me re:
>> state management, hopefully they can clarify/correct my answer :)
>>
>> Regards,
>> Matt
>>
>> > On Jul 28, 2016, at 3:21 PM, Peter Wicks (pwicks) <[hidden email]>
>> wrote:
>> >
>> > I'm using the QueryDatabaseTable processor.  Frequently I would like to
>> set an initial state value for my columns max value, or I'd like to change
>> the value stored in State Management for that column, this comes up in
>> operational scenarios ("reload all data after this exact timestamp, that's
>> when everything went bad").
>> >
>> > But in looking through the code, and even the rest API, I don't see any
>> options to set/modify a processes state.  Am I missing something? Is there
>> a way to set initial values like I want?
>> >
>> > Regards,
>> >  Peter
>>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: State Management - Set initial State or update State through GUI or API

Mel J
Hi,

Just checking if there was a resolution to this . I also have a similar use case to implement "I would like to
set an initial state value for my columns max value, or I'd like to change
the value stored in State Management for that column, this comes up in
operational scenarios ("reload all data after this exact timestamp, that's
when everything went bad").
 But in looking through the code, and even the rest API, I don't see any
 options to set/modify a processes state.  Am I missing something? Is there
 a way to set initial values like I want? "

Thanks
Loading...