Talking to a TCP Socket in XPA #tcp #socket #xpa


Philip Almond
 

Hi all,

I was a Magic developer 20+ years ago and taking a new look at the product and have downloaded a 'home' XPA 3.2.

I have a project that requires receiving data from a TCP Socket. e.g. 192.168.0.101:10000

The data is CRLF delimited and an example of the data is below.

aaA30000000003340001190602063641274eFS
aaA30000000003340001190602063641274eBS
aaA500000000033400011906020636412c7cLS

I want to be able to open a TCP socket, and then have a background process listening for data to arrive. This will be asynchronous. When data arrives it will simply insert it into a table, for subsequent processing. Can this background process be 'interrupt' driven or will it have to poll the buffer?

I see from a post 3 years ago that Steven Blank said "In Magic xpa (either v2.5 or v3.1), you can call System.Net.Sockets directly - no need for an external C# program"

Does anyone have some sample magic 'code' that demonstrates how to implement this? As I say, I have a lot of catching up to do!

Thanks in advance for any help

Philip Almond



Peter Ashworth
 

Hi Philip

I would imagine you could call a batch program, go in to its task properties and on the advanced tab set parallel execution, and maybe single instance as well. This would make it a background process.

Then inside that program create a line in the logic view and create an invoke .net line. Then inside the invoke .net  window you can write the relevant C# code to do the socket communication. Theres "no need for an external C# program" because you can now invoke and write C# code inside magic itself, you still have to write the c# code however. I presume googling c# socket handling would provide a suitable method of this. Also the assemblies in the "using" section of any c# code you write has to be added in to the component repository.

I think there is also a way of raising .net events inside the invoke snippet and capturing/handling them them like a normal event handler would in the parent magic task, and therefore run magic code to insert the data into a table. However I have never got around to that myself beyond handling a .net controls "changed" event on a custom user control I was displaying on a form.

No code samples I'm afraid but hopefully this is a starting point.

Alternatively as a completely separate approach, if you don't mind an external program running, you can use external events to get an external program to pass data to magic and inside the external event handler have the insert to table logic. See http://kb.magicsoftware.com/articles/bl_Reference/External-Event-xpa for more.

Hope this helps

Peter


Andreas Sedlmeier
 

I think the external events do not exist anymore - since quite a while.

I see from a post 3 years ago that Steven Blank said "In Magic xpa (either v2.5 or v3.1), you can call System.Net.Sockets directly - no need for an external C# program"
When you need to send some data over a socket or receive from it (blokcking) you indeed could paste the code in a .NET sniplet and it will work. You just wouldnt because: Magic only supports C# language specification v4 from I think 15 years ago and b) you will have a very hard time to debug this and this kind of stuff you will definitely have to debug once a while. 

You do however want an asynchronouus listener and that works with events and async stuff which you do not have available in  Magic snipletos, ...

Short: Here is the code to implement an async tcp listener: https://docs.microsoft.com/en-us/dotnet/framework/network-programming/asynchronous-server-socket-example
Basically its copy paste to Visual Studio and compile and you can use it from Magic.

Now you only need to raise an event when you receive data which you can handle in Magic. That's relatively easy with C# too. I use it here (https://github.com/asedl/XpaDemo/blob/master/external/Dotnet/XpaTQueue/XpaTQueue/SimpleTopicSubscriber.cs) to inform Magic Xpa if a asynchronous message for a subscription in Apache MQ arrived.

That "intermediate step" I would (strongly) recommend for your requirement too anyways. Let the tcp/ip listener write a message to a MQ and then raise an event to tell Magic that a new message arrived. That adds, without any effort basically guaranteed delivery and transactions (if you need). Besides you have now all the nice monitoring tools and options available, ...

Best regards,

Andreas


 


Peter Ashworth
 

Hi Andreas

I think the external events do not exist anymore - since quite a while.
The external events are still around. Our main app is Magic 9 but we keep doing test conversions and are close to jumping to XPA. In 3.3d our external event still seems to be going strong. It was a contractor who did our dll we use it in so I don't know a lot about it but the only thing we had to get him to tweak was how he identified the right process to send the message to.

Regards

Peter


Andreas Sedlmeier
 

Hi Peter,

Thanks for the info. I swore that I did look for it not so long ago and did not find it. I came to the conclusion that its maybe considered a "enterprise feature" and simply does not exist in Single user edition (which is what I have). I did however check right now, and found it immediately. Its indeed still there.

That's an handy integration option . Messaging protocol is however "send and pray" because Magic will not notify you (like with a return code) if it actually did get the message/event and processed succesfully. Besides I think there's some security implications where one process cannot simply send a message to another and in background mode I think it does not work at all. I knew all these things, its just more than a decade ago ...

Andreas


Peter Ashworth
 

Hi Andreas

Ah single user edition, that might have indeed been different. 

Fair point on the send and pray, never knew much about that end. Our dll is a custom GUI chart, and we use the external event to inform magic that something was clicked on the chart and an ID number. So I suppose if nothing happens the user just clicks again and it succeeds the second time.

In the external message scenario my thought would be put the external event handler in the main program set to sub task enabled. No need to be in a background process as such just needs to be sneaky and clean running batch code in the main program.

But as you say confirmation of receipt/success would probably be best

Hypothetically depending on the database you are using Philip, if you are considering writing a seperate C# program anyway you could even conisder just writing to the database directly and not worry about communicating via magic at all? Dunno if thats any more/less complicated/work.

Regards

Peter


Todd Baremore
 

Philip,

I created something similar using Nodejs that runs on a Windows 2016 server.   It listens for incoming vehicle location data and inserts the data into a MySQL table. There is also a Node data connector for SQL Server.  My requirement was for UDP.  TCP shouldn't be much different.

These will get you started
https://www.youtube.com/watch?v=fBNz5xF-Kx4
https://nodejs.org/en/
https://www.hacksparrow.com/node-js-udp-server-and-client-example.html
https://nodejs.org/api/dgram.html
https://www.w3schools.com/nodejs/ref_dgram.asp
https://www.w3schools.com/nodejs/nodejs_mysql_insert.asp

Todd
On 6/5/2019 4:03 AM, philip.almond@... wrote:

Hi all,

I was a Magic developer 20+ years ago and taking a new look at the product and have downloaded a 'home' XPA 3.2.

I have a project that requires receiving data from a TCP Socket. e.g. 192.168.0.101:10000

The data is CRLF delimited and an example of the data is below.

aaA30000000003340001190602063641274eFS
aaA30000000003340001190602063641274eBS
aaA500000000033400011906020636412c7cLS

I want to be able to open a TCP socket, and then have a background process listening for data to arrive. This will be asynchronous. When data arrives it will simply insert it into a table, for subsequent processing. Can this background process be 'interrupt' driven or will it have to poll the buffer?

I see from a post 3 years ago that Steven Blank said "In Magic xpa (either v2.5 or v3.1), you can call System.Net.Sockets directly - no need for an external C# program" Does anyone have some sample magic 'code' that demonstrates how to implement this? As I say, I have a lot of catching up to do!

Thanks in advance for any help

Philip Almond




Philip Almond
 

Thank you all for your willingness to help.

It may take a while, but I will let you know how I get on. 

Philip


Craig Martin
 

For me, Magic has always been a great manifestation of the 80/20 rule.

And that can be stated in various ways.

Magic with 20% of effort can give you 80% of results, but that final 20% might take another 80% of time or might not even be possible with Magic alone.
At various times, I've advocated rounding out Magic applications with a variety of Delphi, C#, Javascript, NodeJS or whatever,
Now I reckon Python is the Holy Grail - the Python socket lib will give you everything you need here.

And when the Magic developer market eventually dwindles to zero, if it's not already there, Python will still be strong. It's not fake news.
Make choices you can live with. 


From: main@magicu-l.groups.io <main@magicu-l.groups.io> on behalf of philip.almond@... <philip.almond@...>
Sent: Wednesday, June 5, 2019 6:26 PM
To: main@magicu-l.groups.io
Subject: Re: [magicu-l] Talking to a TCP Socket in XPA #TCP #Socket #XPA
 
Thank you all for your willingness to help.

It may take a while, but I will let you know how I get on. 

Philip


Philip Almond
 

Thanks for your help Craig,

I was concerned about your last statement though, so I took a look at their financials for 2018/17 and the last quarter, 2019. I'd be happy if my business was growing at 10-12% in the current economic climate. Revenues are nothing to be sneezed at either.

I do take your point however about being boxed into a corner, but they seem to have it covered well enough (at least for what I need) with extensiblity into VB and C# to some extent. I assume, given Python's #5 position (I Goggled it) that they would be unwise to ignore it in the future???

Cheers
Philip

Financial Highlights for the First Quarter Ended March 31, 2019 • Revenues for the first quarter ended March 31, 2019 increased 3% to $71.8 million compared to $69.7 million in the same period last year. • Operating income for the first quarter ended March 31, 2019 increased 11% to $8.4 million compared to $7.6 million in the same period last year. • Non-GAAP operating income for the first quarter ended March 31, 2019 increased 4% to $10.1 million compared to $9.7 million in the same period last year. • Net income attributable to Magic's shareholders for the first quarter ended March 31, 2019 increased 17% to $5.4 million, or $0.11 per fully diluted share, compared to $4.6 million, or $0.10 per fully diluted share in the same period last year. • Non-GAAP net income attributable to Magic's shareholders for the first quarter ended March 31, 2019 increased 8% to $6.7 million, or $0.14 per fully diluted share, compared to $6.2 million, or $0.14 per fully diluted share, in the same period last year. • Cash flow from operating activities for the first quarter ended March 31, 2019 amounted to $10.7 million compared to $7.3 million in the same period last year. • As of March 31, 2019, Magic’s net cash, cash equivalents, short and long-term bank deposits and marketable securities, offset by financial liabilities, amounted to $80.9 million. 

• Magic is reiterating its fiscal year 2019 guidance issued in February for full year revenues of between $313 million to $319 million on a constant currency basis, reflecting annual growth of 10% to 12%.   



Virus-free. www.avg.com


On Thu, 6 Jun 2019 at 14:49, Craig Martin <craig@...> wrote:
For me, Magic has always been a great manifestation of the 80/20 rule.

And that can be stated in various ways.

Magic with 20% of effort can give you 80% of results, but that final 20% might take another 80% of time or might not even be possible with Magic alone.
At various times, I've advocated rounding out Magic applications with a variety of Delphi, C#, Javascript, NodeJS or whatever,
Now I reckon Python is the Holy Grail - the Python socket lib will give you everything you need here.

And when the Magic developer market eventually dwindles to zero, if it's not already there, Python will still be strong. It's not fake news.
Make choices you can live with. 

From: main@magicu-l.groups.io <main@magicu-l.groups.io> on behalf of philip.almond@... <philip.almond@...>
Sent: Wednesday, June 5, 2019 6:26 PM
To: main@magicu-l.groups.io
Subject: Re: [magicu-l] Talking to a TCP Socket in XPA #TCP #Socket #XPA
 
Thank you all for your willingness to help.

It may take a while, but I will let you know how I get on. 

Philip



--

Trident RFID  
Philip Almond CEO
 
Mobile: +61 422 813 896 Skype: philip.almond62
 
Tridentrfid.com Like us on Facebook

Virus-free. www.avg.com


Steven Blank
 

Philip,

Sorry, late to the party. Apologies.

I just uploaded some demo programs to this forum's shared Files area that illustrate, among other things, the steps required to implement a TCP Listener, without code snippets. Basically, I believe you just need to set up a few variables in Magic and let your listener do, what it do. It's really very straightforward. The programs are online (rather than batch) and contain a ton of error handling – DNException() – as illustration. But, hoprfully, it'll be enough to start you. Suggest you check it out.

You can access this file at the URL:
https://magicu-l.groups.io/g/main/files/Demo_SocketsDotNet.zip

I, of course, provide these simply as free educational aids and offer no promises or guarantees about anything. Whatever. You know the drill.

Cheers,

Steve Blank

On 6/5/2019 1:03 AM, philip.almond@... wrote:
Hi all,

I was a Magic developer 20+ years ago and taking a new look at the product and have downloaded a 'home' XPA 3.2.

I have a project that requires receiving data from a TCP Socket. e.g. 192.168.0.101:10000

The data is CRLF delimited and an example of the data is below.

aaA30000000003340001190602063641274eFS
aaA30000000003340001190602063641274eBS
aaA500000000033400011906020636412c7cLS

I want to be able to open a TCP socket, and then have a background process listening for data to arrive. This will be asynchronous. When data arrives it will simply insert it into a table, for subsequent processing. Can this background process be 'interrupt' driven or will it have to poll the buffer?

I see from a post 3 years ago that Steven Blank said "In Magic xpa (either v2.5 or v3.1), you can call System.Net.Sockets directly - no need for an external C# program" Does anyone have some sample magic 'code' that demonstrates how to implement this? As I say, I have a lot of catching up to do!

Thanks in advance for any help

Philip Almond



Philip Almond
 

Hi Steve,

I imported and ran this and it worked. I tested with Hercules Terminal. 

I need the 'Client' side, rather than the Server side, but should be able to make progress with this as an example.

Much appreciated!

image.png

Regards
Philip

On Thu, 6 Jun 2019 at 15:42, Steven Blank <sgblank@...> wrote:

Philip,

Sorry, late to the party. Apologies.

I just uploaded some demo programs to this forum's shared Files area that illustrate, among other things, the steps required to implement a TCP Listener, without code snippets. Basically, I believe you just need to set up a few variables in Magic and let your listener do, what it do. It's really very straightforward. The programs are online (rather than batch) and contain a ton of error handling – DNException() – as illustration. But, hoprfully, it'll be enough to start you. Suggest you check it out.

You can access this file at the URL:
https://magicu-l.groups.io/g/main/files/Demo_SocketsDotNet.zip

I, of course, provide these simply as free educational aids and offer no promises or guarantees about anything. Whatever. You know the drill.

Cheers,

Steve Blank

On 6/5/2019 1:03 AM, philip.almond@... wrote:
Hi all,

I was a Magic developer 20+ years ago and taking a new look at the product and have downloaded a 'home' XPA 3.2.

I have a project that requires receiving data from a TCP Socket. e.g. 192.168.0.101:10000

The data is CRLF delimited and an example of the data is below.

aaA30000000003340001190602063641274eFS
aaA30000000003340001190602063641274eBS
aaA500000000033400011906020636412c7cLS

I want to be able to open a TCP socket, and then have a background process listening for data to arrive. This will be asynchronous. When data arrives it will simply insert it into a table, for subsequent processing. Can this background process be 'interrupt' driven or will it have to poll the buffer?

I see from a post 3 years ago that Steven Blank said "In Magic xpa (either v2.5 or v3.1), you can call System.Net.Sockets directly - no need for an external C# program" Does anyone have some sample magic 'code' that demonstrates how to implement this? As I say, I have a lot of catching up to do!

Thanks in advance for any help

Philip Almond




--

Trident RFID  
Philip Almond CEO
 
Mobile: +61 422 813 896 Skype: philip.almond62
 
Tridentrfid.com Like us on Facebook

Virus-free. www.avg.com


Andreas Sedlmeier
 

I do take your point however about being boxed into a corner, but they seem to have it covered well enough (at least for what I need) with extensiblity into VB and C# to some extent. I assume, given Python's #5 position (I Goggled it) that they would be unwise to ignore it in the future???
Much of the success of Python comes from IoT and embedded. That's simply not the playground of Magic and it will never be.

I used it last year once a while too, for stuff quite similar to what you originally asked for, IoT gateways which collect data from dumb devices or get from smart and forward it to the cloud. Thats indeed not very hard with Python, you will find a lot of existing code, have the Rasperry Pi, ...

Before you however  do not write what you actually need (an asynchronous server like in the beginning), or just a "tcp client" which sends some data and disconnects, nobody can really help you.

Asycnchronoous, none-blocking, communication is not possible with Sockets in Magic, it has no callbacks and those you need, for read / write and accept.  Steve showed blocking and synchronous in his sampole. I just had a look.

Best regards,

Andreas