MS SQL CDC Processor

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

MS SQL CDC Processor

Viresh R Navalli
Hi Team,

We are looking into CDC for MS SQL, got reference to below link.
https://github.com/apache/nifi/pull/2231

I have checked in nifi-1.9.2 release didn't see such processor. Is this functionality available in latest release.

_________________
Thanks & Regards
Viresh N

============================================================================================================================

Disclaimer:  This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at http://www.techmahindra.com/Disclaimer.html <http://www.techmahindra.com/Disclaimer.html> externally http://tim.techmahindra.com/tim/disclaimer.html <http://tim.techmahindra.com/tim/disclaimer.html> internally within TechMahindra.

============================================================================================================================
Reply | Threaded
Open this post in threaded view
|

RE: [EXT] MS SQL CDC Processor

Peter Wicks (pwicks)
Viresh,

There are a couple of options for MS SQL CDC.  The "simple" option is to create a view in MS SQL that joins the CDC table with the lsn_time_mapping table. Then use the QueryDatabaseTable processor to load this view incrementally using the "tran_end_time" column.

I see you've also found my pull request from October 2017. I put a lot of time into getting this working and covering all of my use cases, unfortunately, there are very few users available to test. Because of the lack of testing, this PR has been hanging out for quite a while now.  MS SQL CDC used to be an Enterprise edition only feature, and only in recent years has become available in all versions. I'd still like to see this PR get merged in the future, if users become available for testing.

Thanks,
  Peter

-----Original Message-----
From: Viresh R Navalli <[hidden email]>
Sent: Wednesday, April 17, 2019 12:41 AM
To: [hidden email]
Cc: Chandrashekhar Thakare <[hidden email]>
Subject: [EXT] MS SQL CDC Processor

Hi Team,

We are looking into CDC for MS SQL, got reference to below link.
https://nam01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fnifi%2Fpull%2F2231&amp;data=02%7C01%7Cpwicks%40micron.com%7C3a87b523908c481272e608d6c3276c09%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636910971434384147&amp;sdata=byIejgwoWBWf%2F9udig411H7k4%2Bszm8nYLL%2BxIb2xm5Y%3D&amp;reserved=0

I have checked in nifi-1.9.2 release didn't see such processor. Is this functionality available in latest release.

_________________
Thanks & Regards
Viresh N

============================================================================================================================

Disclaimer:  This message and the information contained herein is proprietary and confidential and subject to the Tech Mahindra policy statement, you may review the policy at https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.techmahindra.com%2FDisclaimer.html&amp;data=02%7C01%7Cpwicks%40micron.com%7C3a87b523908c481272e608d6c3276c09%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636910971434384147&amp;sdata=hzVhOkqXAawtnLQGVvSZIwXLF%2FD9p%2FKT%2BPUHeLpDfTU%3D&amp;reserved=0 <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.techmahindra.com%2FDisclaimer.html&amp;data=02%7C01%7Cpwicks%40micron.com%7C3a87b523908c481272e608d6c3276c09%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636910971434384147&amp;sdata=hzVhOkqXAawtnLQGVvSZIwXLF%2FD9p%2FKT%2BPUHeLpDfTU%3D&amp;reserved=0> externally https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftim.techmahindra.com%2Ftim%2Fdisclaimer.html&amp;data=02%7C01%7Cpwicks%40micron.com%7C3a87b523908c481272e608d6c3276c09%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636910971434384147&amp;sdata=VRreLAKYUieyYwp58pQuXCdVy4XyNpQOKD04NqwRmDE%3D&amp;reserved=0 <https://nam01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftim.techmahindra.com%2Ftim%2Fdisclaimer.html&amp;data=02%7C01%7Cpwicks%40micron.com%7C3a87b523908c481272e608d6c3276c09%7Cf38a5ecd28134862b11bac1d563c806f%7C0%7C0%7C636910971434384147&amp;sdata=VRreLAKYUieyYwp58pQuXCdVy4XyNpQOKD04NqwRmDE%3D&amp;reserved=0> internally within TechMahindra.

============================================================================================================================