Oracle OSB 12c – File Polling/Reading and Writing using File Transport

July 1, 2015 SJ SOA-OSB-12c 4

In this article, we will explore the use of File Transport node in OSB 12c for file polling/reading and writing to local machine

1. Business case requirement :

  • Read or poll files periodically from local machine and write to other location in the same local machine (this is just for demo propose)

2. Assumptions :

  • All file reading/polling and writing are in the same local machine

Let’s get started with designing OSB module for File Polling/Reading and Writing using File Transport example in Oracle JDeveloper 12c IDE

3. File Polling/Reading & Writing using File Transport :

Step 1: Open New Project

File Menu –> New –> Project

1_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_New_Project

Step 2: Select Service Bus Tier from available categories

Service Bus Tier –> Service Bus Project –> OK

This opens up wizard for New Service Bus Project

2_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_New_Service_Bus_Project

Note: Comparing this wizard with the one we have created in this article, the major difference is “Service Bus Project” option is enabled since we have one Service Bus Application in the user/developer workspace (already created)

Step 3: Enter Project Name and its directory location for the new Service Bus Project

Leave the default values for demo; otherwise we can give more specific names in accordance with naming convention followed in the project

Project Name: FilePollReadAndWriteUsingFileTransportSB

Click Finish

3_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_New_Service_Bus_project_name

New Service Bus project will be created with required artifacts

Step 4: Create required folders in accordance with project standards (naming conventions)

Generally, there should be

  • ProxyService folder to place/store/save the proxy services which ends with .proxy
  • BusinessService folder to place/store/save the business services which ends with .bix
  • Resources folder to store all other required artifacts like WSDL, Schema and transformations files

Now, we will create above listed folders

Right click on the Service Bus project –> New –> Folder…

Opens up a new wizard

4_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_create_new_folder

Note: If you don’t find “folder” option directly, then click on “From Gallery” and from there choose folder option and continue with below steps

Create Folder wizard –> enter folder name like, for example “BusinessService” for Business Services

Finally click OK –> to create folder

5_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_new_folder_name

Similarly, we can create “ProxyService” folder

After creating two folders namely ProxyService and BusinessService, our project will look like below screen capture

6_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_after_all_folders_create_steps

3.1 Business Service :

Step 5: Create Business Service for file writing

Drag and drop “File Transport” component from Technology palate under advanced category to External Services lane (Business Service in 11g)

7_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Ext_Services

Upon dropping, a new wizard pops up to enter the required details for Business Service or External Service

Step 5.a: Fill up mandatory details for Business Service (i.e .bix file)

Service Name: FileWritingBS (append BS to the business service as a naming standard)
Location: browse through “BusinessService” folder we created in earlier step
Transport: file (keep it as it is)

Note: There are no other options available under Transport drop-down other than File Transport (file protocol)

Click Next

8_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Ext_Services_5a

Step 5.b: In step 2 of Create Business Service wizard –> set “Request” & “Response” Messaging type for the File Writing service

Select Text for Request Messaging type and None for Response Messaging type

Click Next

9_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Ext_Services_5b

Note: The response message type must be None because the File transport supports only one-way messaging

Step 5.c: In step 3 of Create Business Service wizard –> set Endpoint URI to local file server location à where we target to write the files

In this example, “D:\REPOSITORY\FileWriteServerLocation” but it can be any valid location on the local machine for File Transport

Note: For remote server or FTP server locations, use FTP Transport instead of File Transport. In the next article, we will demonstrate the same example using FTP Transport

Click Finish

10_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Ext_Services_5c

Finally, File Transport Business Service is created at the External Services lane

11_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Business_Service_created

Optionally, set File Transport Protocol details –> Save all

This is useful to set prefix and suffix of the filename that is going to be created. Still, we don’t have full control over the filename because it will auto generate a name with random logic and sandwich between the prefix & suffix we supplied in the Transport Details tab

Double-click on “FileWritingBS” from Service Bus overview or else open “FileWritingBS.bix” file under BusinessService folder

Go to “Transport Details” tab –> Set “Prefix” and “Suffix”

Prefix: DataFile_

Suffix: .txt

12_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Business_Service_setting_file_protocol_details

3.2 Proxy Service :

Step 6: Create Proxy Service and add pipeline for file polling or reading

Drag and drop “File Transport” component from Technology palate under advanced category to Proxy Services lane

13_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Proxy_Services

Upon dropping, a new wizard pops up to enter the required details for Proxy Service

Step 6.a: Fill up mandatory details for Proxy Service (i.e .proxy file)

  • Service Name: FilePollAndReadPS (append PS to the proxy service as a naming standard)
  • Location: browse through “ProxyService” folder we created in earlier step
  • Transport: file (keep it as it is)

Note: There are no other options available under Transport drop-down other than File Transport (file protocol)

Keep “Generate Pipeline” checked for creating pipeline along with Proxy Service

Click Next

14_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Proxy_Services_6a

Step 6.b: In step 2 of Create Proxy Service wizard –> set “Request” & “Response” Messaging type for the File polling or reading service

Select Text for Request Messaging type and None for Response Messaging type

Click Next

15_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Proxy_Services_6b

Note: The response message type must be None because the File transport supports only one-way messaging

Step 6.c: In step 3 of Create Proxy Service wizard –> set Endpoint URI to local file server location –> from where this service going to poll or read files periodically

In this example, “D:\REPOSITORY\FilePollReadServerLocation” but it can be any valid location on the local machine for File Transport

Note: For remote server or FTP server locations, use FTP Transport instead of File Transport. In the next article, we will demonstrate the same example using FTP Transport

Click Finish

16_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_file_transport_to_Proxy_Services_6c

Finally Proxy Service is created for File Reading/Polling

Proxy Service is created on the Proxy Services lane and this resides under “ProxyService” folder

17_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Proxy_Services_created

Error: This is due to; we are yet to set couple of mandatory attribute values. Once after setting these mandatory values & saving, all errors will vanish

Let’s do that !!

Step 7: Double-click on “FilePollAndReadPS” from Service Bus overview or else open “FilePollAndReadPS.proxy” file under ProxyService folder

Go to “Transport Details” tab –> set “Stage Directory*” and “Error Directory*”

  • Stage Directory* –> D:\REPOSITORY\FilePollReadServerLocation\stage
  • Error Directory* –> D:\REPOSITORY\FilePollReadServerLocation\error
  • File Mask* –> *.txt (default value is *.* which means it can poll or read any files from “D:\REPOSITORY\FilePollReadServerLocation” location but we have restricted to poll/read only text files by setting the value to *.txt)

Optionally, we can set archive directory for file archival post reading the files but we have to make sure to select “Archive” from Post Read Option*

  • Archive Directory* –> D:\REPOSITORY\FilePollReadServerLocation\archive

Other attributes we can keep as it is or else fine tune depending upon the business requirement

18_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Proxy_Services_setting_mandatory_values

Step 8: Double-click on the FilePollAndReadPSPipeline.pipeline from Service Bus overview which opens up in a new editor with .pipeline extension (green-color round-cornered rectangle type figure in the Pipelines/Split Joins)

Opens up FilePollAndReadPSPipeline.pipeline file in an editor

19_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Service_Bus_Overview

Opens up in a new pipeline editor

20_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Proxy_Service_pipeline_editor

Step 9: Routing node

Drag and drop “Routing” from “Message Flow” under category “Route”

3.3 Advantages of OSB 12c over OSB 11g :

  • When we drag and drop the Routing or Routing Table node, it automatically have a Route node in place
  • Whereas in 11g, we need to drag and drop Route node first before Routing or Routing Table node
21_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_drag_drop_Routing_node_to_pipeline

Error: Red-ribbon type error mark on top-right corner is because; we haven’t set correct mandatory values for Routing node (after setting correct values & saving would vanishes all errors)

Step 10: Add business service to Routing node

Click on Routing –> opens corresponding properties in the dialog box below

Click browse icon next to Service* –> opens “Resource Chooser” –> select service to be routed

Finally Click OK

22_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_set_business_service_in_Routing_node

Save All –> all error will be erased. Time for testing

Step 11: Time to test !!

Testing Service Bus project –> deploying into IntegratedWeblogicServer

23_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_testing_Run

Step 12: Testing

This will open our traditional OSB console

24_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_traditional_osb_console

Note: But we don’t need to execute, supplying any inputs for file polling/reading & writing

Just deploy the Service Bus application into IntegratedWeblogicServer and it start polling/reading the files from poll/read location periodically that is set to 1 minute (60 seconds)

We have restricted to poll/read only text files that is .txt files (Text Documents)

Poll/Read location: D:\REPOSITORY\FilePollReadServerLocation

25_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_ReadPoll_location

After polling/reading, file temporarily moves to stage directory for processing

Stage Directory: D:\REPOSITORY\FilePollReadServerLocation\stage

26_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_stage_directory

Note: Since the file size of Text Document is 1 KB, processing happens in a fraction of second. So we can’t actually see files moved to this location. Same is not true for large size files

After processing files in the stage directory, files are permanently moved to file write location i.e.;

File writing location: D:\REPOSITORY\FileWriteServerLocation

27_OSB-12c_File_Polling_Reading_Writing_using_File_Transport_Write_location

Related Articles :

References :

Happy Coding !!
Happy Learning !!

Oracle OSB 12c - File Polling/Reading and Writing using FTP Transport and authentication via Service Account
Oracle OSB 12c - Java Callout and Routing Table example

Related posts:

  1. Oracle SOA 12c – Domain creation steps
  2. Oracle OSB 12c – Hello World mock service with Proxy Service
  3. Oracle OSB 12c – Hello World service with both Business and Proxy Service
  4. Oracle OSB 12c – File Polling/Reading and Writing using FTP Transport and authentication via Service Account