This is the first job of a serie.
I need support in filtering data. I have about 70 million values (tick data) from EURUSD cross. Each row has date, time and price value.
- i need a script, program, formula or anything else that allows me to do what is described below. I'm open for your comments and advices. Possibly, if a program, it should be free or easy to use. I have excel, but i fear 70 mio. values are too much.
- i want to filter data similar to how data are filtered for point and figure or renko charts. Only increases or decreases of x pips (price) should be considered, all other data can be deleted.
- starting from the first price value , the next value greater or equal than first value + n.x pips (resp. the next value lower or equal than first value - n.x pips) should be considered. Values in between should be eliminated. --> see sheet2 of excel attached
- i should be able to change the x value
- to each increase of price (of the filtered data), a +1 should be attributed, while to each decrease of data, a -1 should be attributed.
To better understand the concept, see sheet 2 in eattached excel. Also find some example data in sheet 1 for your tests.
Hello, filtering 70M values is a lot, but doable. I will code a program(desktop PC based), that will filter the results. The X value will be changeable. I'll need several days for testing and optimizing the software. If you are interested in my bid, please contact me. Thank You. Best Regards.
Hey There,
Thanks for posting the project. Its looks feasible and am Interested to do it.
Next steps:
Lets discuss/validate the complete requirement and I can start to get this done with required quality output.
I am an excel/access VBA automation professional having 7+ years of experience and can do it with the required quality.
Lets discuss more online on chat.
Thanks,
Abhinav
Hi,
I can do this for you. I will write it in Python, which is freely available on all operating systems. It should take no more than two days and I will provide documented code along with instructions on how to use it, and I am always available to help.
As bonus material I can plot the resulting data set.
I look forward to working with you.
Regards,
Mark
Hi,
I can write a little program in java to filter the data you need.
You need only give the path to the file with the data and the pips value you want and the program will create a file with the data filtered.
Kind greetings
Hi,
I am very experienced in Algorithm Design as a former informatics Olympiad contestant and current contest programmer. Review my profile and let me know if you are interested. I can do this easily and quickly. Payment is up to discussion.
Thanks
I am a very good and intellectual report writer as well as an efficient academic writer.
I am well acquainted with the kind of work you need to be done .
I assure you to complete the work with complete informations required in the project and also submit that in the required time your are supposed to receive.
I have read the needs of your project thoroughly and is completely comfortable with the requirements needed by you.
you need not bother for the completeness and time limits for the work at which i am best careful and successful
I have never put down the belief of persons who hired me earlier and is very sure to retain my record and greet you also with my excellent work and efficiency.
Hi
This is the first contract I have bid for on this site so forgive me if I don’t provide all the details one would normally expect to see.
It sounds to me like your requirement is conceptually very straightforward, the only difficulty being the volume of data, I would therefore propose that we import the data into a database product (say Microsoft Access) and replicate the filters you already have in Excel. I have created a database of 70m records in Access and it is within the size limits (1.2GB against a limit of 2GB)
Why should you award me this contract? Because I am very overqualified to complete this project yet am willing to do the work for a fraction of my day rate in order to get my first project completed on Freelancer.com. Being British there is also likely to be fewer language issues.
Best regards
Jonathan
Hi
I have read your job post and very interested in this. I am experienced data miner and preproccessor and usually i used Rapid miner or weka for the processing. I am available for this job. I am also computer science graduate from very prestigious institute of my country.
Let me know what you think?
Looking forward to hear from you soon.
Thanks
Faisal Riaz
Hello,
I carefully read your requirements and decide to take the project.
About my profile, I was studying business information system, created many Business Intelligent system with SAP Crystal Dashboard, Web applications for data mining. I am now developing a BI System to filter thousand of record about the domain name. I recently interested in Big Data and I want to challenge myself with 70 millions records from your data.
As I am a full-time staff, I can only work for several hours at the weekend (Saturday and Sunday). I plan to finish your requirement by developing a web application using Ruby on Rails and MongoDB (NoSQL database for big data). The project may vary from 2 to 4 weeks due to my working limitation. I would prefer a close connection via skype or email to clearly understand and quickly adapt to any of your ideas.
Please contact me if you need further discussion.
I hope we can work together for the long-term.
Thanks
Hi,
We are a BAI (Business Analytics and Intelligent) company.
We specialized in
Data Integration
Data Processing
Data Warehouse
Business Intelligence
Visualization
Big Data (Hadoop)
------------------------------
We propose to use a tool to do that.
Our proposal is we will do it using open source ETL tool (like Pentaho or Talend).
Work will include:
Creating the code.
Setting up the ETL tool + Directory in one machine.
Quick help file to run the process.
Process:
You may run the process by following instruction.
To change x value, change the value in a reference file before run.
Looking forward.
