Having the ability to disable controlling mechanism for agents

[Marc Parisi]

Good Morning/Afternoon/Evening Everyone,

   A pull request exists to introduce some very limited command and control
capabilities. In it one of the developers questioned why we were attempting
to send a heartbeat to a command and control server when it was not
configured. That's a very valid point.  By design we have a C2 agent that
uses a protocol to transmit/receive messages and/or commands. If the
protocol isn't configured we're effectively neutered, but we still have the
ability to have local control through scripts or executables, so I would
like to see C2 a permanent aspect of the MINIFi CPP agent that we have.

   If protocol endpoints aren't configured as the reviewer noted we
shouldn't be attempting them, so that point is settled ( as in he is
correct ), but the penultimate question is whether or not we should allow
C2 to be disabled by design. Should this feature be pluggable to the agent?
My initial design was that it should not since allowing command and control
to be disabled appeared antithetical with its intent; however, I'd love to
hear other input as I may be entirely off base ( sometimes I'm not even on
the right ball field!).

  Given the previous sentiment that a C2 agent could be controlled locally,
and the capabilities can be used for localized debug/control, should we
ever allow C2 to disabled from the default agent? I acknowledge that others
may build their own agents without command and control capabilities, so I
can only ask this question in regards to the agent that we are building and
including into the Apache MiNiFi CPP project.

  TL;DR : Should we allow C2 capabilities to be disabled?


  Thank for your time and consideration,
  Marc

[Aldrin Piri]

My feel is that there should be a way to enable/disable C2 as the user sees
fit.

There may be other implementations that take their own approach but I can
see many users taking the one that is currently in the codebase as it may
cover the majority of their needs.  I could imagine that some would be
averse to having that functionality "always on" but would still appreciate
the option of the capability when they wanted to dig in after the fact and
are okay with a config change (and restart if needed).

On Tue, Sep 26, 2017 at 9:28 AM, Marc <[hidden email]> wrote:

> Good Morning/Afternoon/Evening Everyone,
>
>    A pull request exists to introduce some very limited command and control
> capabilities. In it one of the developers questioned why we were attempting
> to send a heartbeat to a command and control server when it was not
> configured. That's a very valid point.  By design we have a C2 agent that
> uses a protocol to transmit/receive messages and/or commands. If the
> protocol isn't configured we're effectively neutered, but we still have the
> ability to have local control through scripts or executables, so I would
> like to see C2 a permanent aspect of the MINIFi CPP agent that we have.
>
>    If protocol endpoints aren't configured as the reviewer noted we
> shouldn't be attempting them, so that point is settled ( as in he is
> correct ), but the penultimate question is whether or not we should allow
> C2 to be disabled by design. Should this feature be pluggable to the agent?
> My initial design was that it should not since allowing command and control
> to be disabled appeared antithetical with its intent; however, I'd love to
> hear other input as I may be entirely off base ( sometimes I'm not even on
> the right ball field!).
>
>   Given the previous sentiment that a C2 agent could be controlled locally,
> and the capabilities can be used for localized debug/control, should we
> ever allow C2 to disabled from the default agent? I acknowledge that others
> may build their own agents without command and control capabilities, so I
> can only ask this question in regards to the agent that we are building and
> including into the Apache MiNiFi CPP project.
>
>   TL;DR : Should we allow C2 capabilities to be disabled?
>
>
>   Thank for your time and consideration,
>   Marc
>

[Joe Witt]

I think it is worth making it configurable.  There may be cases where
they are so bandwidth constrained, for example, that they don't want
to spend any bits on sending back heartbeats. In the case heartbeats
are not sent back then we lose information about which commands have
been received.  Though through provenance or data tagging we could
still store flow version information as an in-band.  In general we
should ideally have multiple paths to acquire critical information.

On Tue, Sep 26, 2017 at 4:32 PM, Aldrin Piri <[hidden email]> wrote:

> My feel is that there should be a way to enable/disable C2 as the user sees
> fit.
>
> There may be other implementations that take their own approach but I can
> see many users taking the one that is currently in the codebase as it may
> cover the majority of their needs.  I could imagine that some would be
> averse to having that functionality "always on" but would still appreciate
> the option of the capability when they wanted to dig in after the fact and
> are okay with a config change (and restart if needed).
>
> On Tue, Sep 26, 2017 at 9:28 AM, Marc <[hidden email]> wrote:
>
>> Good Morning/Afternoon/Evening Everyone,
>>
>>    A pull request exists to introduce some very limited command and control
>> capabilities. In it one of the developers questioned why we were attempting
>> to send a heartbeat to a command and control server when it was not
>> configured. That's a very valid point.  By design we have a C2 agent that
>> uses a protocol to transmit/receive messages and/or commands. If the
>> protocol isn't configured we're effectively neutered, but we still have the
>> ability to have local control through scripts or executables, so I would
>> like to see C2 a permanent aspect of the MINIFi CPP agent that we have.
>>
>>    If protocol endpoints aren't configured as the reviewer noted we
>> shouldn't be attempting them, so that point is settled ( as in he is
>> correct ), but the penultimate question is whether or not we should allow
>> C2 to be disabled by design. Should this feature be pluggable to the agent?
>> My initial design was that it should not since allowing command and control
>> to be disabled appeared antithetical with its intent; however, I'd love to
>> hear other input as I may be entirely off base ( sometimes I'm not even on
>> the right ball field!).
>>
>>   Given the previous sentiment that a C2 agent could be controlled locally,
>> and the capabilities can be used for localized debug/control, should we
>> ever allow C2 to disabled from the default agent? I acknowledge that others
>> may build their own agents without command and control capabilities, so I
>> can only ask this question in regards to the agent that we are building and
>> including into the Apache MiNiFi CPP project.
>>
>>   TL;DR : Should we allow C2 capabilities to be disabled?
>>
>>
>>   Thank for your time and consideration,
>>   Marc
>>

[Marc Parisi]

Aldrin ( and Joe ) ,
  Thanks for your response. The transmission mechanism can be disabled and
is configurable. I'm speaking more to being able to configure C2 as a
triage mechanism that can work locally if desired. I don't believe I made
that clear.

  One caveat I did not discuss was that if a developer were to build his or
her agent, they are free to remove this component or make it configurable.

  My thought process was that if we allow C2 to be disabled in the default
agent, we have no path of triage. As I vaguely alluded to in the first
E-mail scripts in a localized environment can be used to piggy back on C2
to provide debug information and control the agent. Requiring these
features gives us a path to help with deployment and control of agents.
DevOps could then make their own scripts if needed to piggy back on C2
capabilities, but allowing it to be disabled may lose this key
functionality.

   I suppose the caveat emptor exists for developers building their own
agents just as much as it would for those disabling C2, but my opinion on
requiring it was attempting to avoid the users who deploy MiNiFI-cpp,
disabling "localized command and control" and have no way of triaging the
problem as it occurs -- or being able to give us information to triage.  Do
either of you ( or anyone else? ) believe this is an unfounded fear since
the user is free to make his or her choice about their deployment?

On Tue, Sep 26, 2017 at 4:32 PM, Aldrin Piri <[hidden email]> wrote:

> My feel is that there should be a way to enable/disable C2 as the user sees
> fit.
>
> There may be other implementations that take their own approach but I can
> see many users taking the one that is currently in the codebase as it may
> cover the majority of their needs.  I could imagine that some would be
> averse to having that functionality "always on" but would still appreciate
> the option of the capability when they wanted to dig in after the fact and
> are okay with a config change (and restart if needed).
>
> On Tue, Sep 26, 2017 at 9:28 AM, Marc <[hidden email]> wrote:
>
> > Good Morning/Afternoon/Evening Everyone,
> >
> >    A pull request exists to introduce some very limited command and
> control
> > capabilities. In it one of the developers questioned why we were
> attempting
> > to send a heartbeat to a command and control server when it was not
> > configured. That's a very valid point.  By design we have a C2 agent that
> > uses a protocol to transmit/receive messages and/or commands. If the
> > protocol isn't configured we're effectively neutered, but we still have
> the
> > ability to have local control through scripts or executables, so I would
> > like to see C2 a permanent aspect of the MINIFi CPP agent that we have.
> >
> >    If protocol endpoints aren't configured as the reviewer noted we
> > shouldn't be attempting them, so that point is settled ( as in he is
> > correct ), but the penultimate question is whether or not we should allow
> > C2 to be disabled by design. Should this feature be pluggable to the
> agent?
> > My initial design was that it should not since allowing command and
> control
> > to be disabled appeared antithetical with its intent; however, I'd love
> to
> > hear other input as I may be entirely off base ( sometimes I'm not even
> on
> > the right ball field!).
> >
> >   Given the previous sentiment that a C2 agent could be controlled
> locally,
> > and the capabilities can be used for localized debug/control, should we
> > ever allow C2 to disabled from the default agent? I acknowledge that
> others
> > may build their own agents without command and control capabilities, so I
> > can only ask this question in regards to the agent that we are building
> and
> > including into the Apache MiNiFi CPP project.
> >
> >   TL;DR : Should we allow C2 capabilities to be disabled?
> >
> >
> >   Thank for your time and consideration,
> >   Marc
> >
>

[Kevin Doran]

Hey Marc,

Thanks for kicking off this discussion.

I think I would be in favor of allowing MiNiFi Agent users to disable C2 should they desire, without excluding it from the build. So long as we're supporting a build configuration that would remove C2, it seems to me it should be feasible to include it in the build but disable it through configuration, and if so, that would be my vote.

I see your point about that potentially leaving an admin or local user at a loss to perform debugging or administrative duties, at least without a restart. To that end, I would add a section to the Admin Guide for the MiNiFi agent detailing how C2 mode can be configured to be utilized for local control only, the downsides of disabling it for local troubleshooting and operation. If, over time, we feel disabling the capability significantly impedes the operational functionality of the agent, we can include a config that enables it for local I/O by default and some explanation of the reduced functionality if it is disabled.

Thanks,
Kevin


On 9/27/17, 09:11, "Marc" <[hidden email]> wrote:

    Aldrin ( and Joe ) ,
      Thanks for your response. The transmission mechanism can be disabled and
    is configurable. I'm speaking more to being able to configure C2 as a
    triage mechanism that can work locally if desired. I don't believe I made
    that clear.
   
      One caveat I did not discuss was that if a developer were to build his or
    her agent, they are free to remove this component or make it configurable.
   
      My thought process was that if we allow C2 to be disabled in the default
    agent, we have no path of triage. As I vaguely alluded to in the first
    E-mail scripts in a localized environment can be used to piggy back on C2
    to provide debug information and control the agent. Requiring these
    features gives us a path to help with deployment and control of agents.
    DevOps could then make their own scripts if needed to piggy back on C2
    capabilities, but allowing it to be disabled may lose this key
    functionality.
   
       I suppose the caveat emptor exists for developers building their own
    agents just as much as it would for those disabling C2, but my opinion on
    requiring it was attempting to avoid the users who deploy MiNiFI-cpp,
    disabling "localized command and control" and have no way of triaging the
    problem as it occurs -- or being able to give us information to triage.  Do
    either of you ( or anyone else? ) believe this is an unfounded fear since
    the user is free to make his or her choice about their deployment?
   
    On Tue, Sep 26, 2017 at 4:32 PM, Aldrin Piri <[hidden email]> wrote:
   
    > My feel is that there should be a way to enable/disable C2 as the user sees
    > fit.
    >
    > There may be other implementations that take their own approach but I can
    > see many users taking the one that is currently in the codebase as it may
    > cover the majority of their needs.  I could imagine that some would be
    > averse to having that functionality "always on" but would still appreciate
    > the option of the capability when they wanted to dig in after the fact and
    > are okay with a config change (and restart if needed).
    >
    > On Tue, Sep 26, 2017 at 9:28 AM, Marc <[hidden email]> wrote:
    >
    > > Good Morning/Afternoon/Evening Everyone,
    > >
    > >    A pull request exists to introduce some very limited command and
    > control
    > > capabilities. In it one of the developers questioned why we were
    > attempting
    > > to send a heartbeat to a command and control server when it was not
    > > configured. That's a very valid point.  By design we have a C2 agent that
    > > uses a protocol to transmit/receive messages and/or commands. If the
    > > protocol isn't configured we're effectively neutered, but we still have
    > the
    > > ability to have local control through scripts or executables, so I would
    > > like to see C2 a permanent aspect of the MINIFi CPP agent that we have.
    > >
    > >    If protocol endpoints aren't configured as the reviewer noted we
    > > shouldn't be attempting them, so that point is settled ( as in he is
    > > correct ), but the penultimate question is whether or not we should allow
    > > C2 to be disabled by design. Should this feature be pluggable to the
    > agent?
    > > My initial design was that it should not since allowing command and
    > control
    > > to be disabled appeared antithetical with its intent; however, I'd love
    > to
    > > hear other input as I may be entirely off base ( sometimes I'm not even
    > on
    > > the right ball field!).
    > >
    > >   Given the previous sentiment that a C2 agent could be controlled
    > locally,
    > > and the capabilities can be used for localized debug/control, should we
    > > ever allow C2 to disabled from the default agent? I acknowledge that
    > others
    > > may build their own agents without command and control capabilities, so I
    > > can only ask this question in regards to the agent that we are building
    > and
    > > including into the Apache MiNiFi CPP project.
    > >
    > >   TL;DR : Should we allow C2 capabilities to be disabled?
    > >
    > >
    > >   Thank for your time and consideration,
    > >   Marc
    > >
    >