How to join the source data having 100 million records?

Newbie003242 Lv1Posted 2023-Jun-03 14:29

In Informatica PowerCenter, the normal join is working for the src file having data of 10000 records, but it's throwing an error for 100 million src records. How should I fix this issue?

I have enabled the sorted port in joiner and have even added a sorter before joiner and have sorted the ports which are the same in both the sorters I have used. I am getting the error as(no ports are sorted)

By solving this question, you may help 81 user(s).

Posting a reply earns you 2 coins. An accepted reply earns you 20 coins and another 10 coins for replying within 10 minutes. (Expired) What is Coin?

Enter your mobile phone number and company name for better service. Go

Faisal Piliang Posted 2023-Jun-04 00:04
  
Hi,

You can use source qualifier SQ than remove joiner and join in DB. If no, can you convert the smaller table to a lookup

Thanks
RegiBoy Lv5Posted 2023-Jun-05 09:29
  
Check the physical connection of the cables. I think it is faulty cables
Kyle Padilla Posted 2023-Jun-05 09:38
  
You may utilize the source qualifier SQ, then delete the joiner and join in the database.
Naomi Posted 2023-Jun-05 09:43
  
Ensure that the server hosting Informatica PowerCenter has enough memory, CPU, and disk space to handle the larger dataset. Insufficient resources can lead to errors and performance issues.
MISMIS Lv3Posted 2023-Jun-05 09:45
  
Increase the memory allocation for the Informatica Integration Service to accommodate the larger dataset. You can modify the Integration Service properties in the Informatica Administrator Console.
Adonis001 Lv3Posted 2023-Jun-05 09:48
  
Check session configuration: Confirm that the session properties are set correctly. Specifically, verify that the "Sorted Input" option is enabled for the source qualifier transformation connected to your source file.
isabelita Lv3Posted 2023-Jun-05 09:54
  
As you mentioned, enabling the "Sorted Input" option in the Joiner transformation is a good step. However, make sure that the ports you are using for joining are indeed sorted. Double-check the sorting options for both the source qualifier
BitCloud Lv3Posted 2023-Jun-05 09:56
  
Ensure that the sort orders of the sorted ports in both the source qualifier transformation and the sorter transformation are the same. If the sort orders differ, you may encounter the "no ports are sorted" error.
Natsu Dragneel Lv3Posted 2023-Jun-05 10:28
  
Increase the memory allocation for the Informatica Integration Service to accommodate the larger dataset.

I Can Help:

Change

Moderator on This Board

4
0
1

Started Topics

Followers

Follow

Trending Topics

Board Leaders