Find Jobs
Hire Freelancers

Network programing in c++ (http filesystem using fuse)

$100-500 USD

In Progress
Posted almost 8 years ago

$100-500 USD

Paid on delivery
There is a FUSE based filesystem called httpFS. Search google for httpfs. I need you to enhance its functionality. There is no need for you to understand how filesystems work, the code enhancement is mainly about sockets and connections. So you need to know how to use sockets in c++. There are currently two tasks to do: Firstly, the httpfs filesystem re-connects for every 4k block of data. Your goal is to modify the code so it can reuse the previously created connection (keep alive) and there needs to be some read-ahead mechanism, so if the filesystem reads 4KB, the actual connection reads eg. 40KB and caches it somewhere in RAM so subsequent reads from consecutive position gets the data right away. Secondly, the httpfs filesystem currently supports one argument - a remote URL for one file, for example URL to an ISO image. I need you to implement a mirrors support, as explained below: When the FUSE filesystem is mounted, it looks like it contains the single file from the URL already downloaded - but the file is not downloaded (can have several gigabytes). Instead, when any application accesses the file, it download the actually read()'ed parts (4K blocks) in background, instead of downloading the whole file. This is already implemented and works. I need you to enhance the code in a way that user will provide three URLs - the URL for iso image like before, but then also an URL which will contain list of alternate URLs of the same file, and third URL which will contain md5 checksums for all 4k blocks of the iso file. With this, the fuse filesystem will then be able to download random parts from random URLs, and check if any given 4k block has a valid checksum. You must keep in mind that: - the code must re-download the list of mirrors periodically from time to time, because it may change every few minutes, so the list of mirrors needs to be updated in memory - the list of md5 checksums will never be updated, so once you download it, there is no need to re-download it again. - any of the URLs in mirrors list may not be accessible after some time, so once it is not accessible, it must remember the URL in some sort of blacklist to never try it anymore - it must keep open several connections at the same time. There must be a connections pool. It will probably never read from different connections at the same time, but it should keep them open and randomly pick from which one it's going to read now. - it must handle properly connection timeouts, to easily detect that any given URL is no longer accessible. Also if a md5 checksum if not verified, it should consider that URL as broken, and blacklist it. You should not mind parallel processing because, as far as I am aware of, FUSE filesystems do not support parallel filesystem access.
Project ID: 10525845

About the project

8 proposals
Remote project
Active 8 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
8 freelancers are bidding on average $530 USD for this job
User Avatar
I am very proficient in c and c++. I have 16 years c++ developing experience now, and have worked for more than 6 years. My work is online game developing, and mainly focus on server side, using c++ under Linux environment. I used c++ to make many great projects, for example, I made the tools which can convert java files into c++ scripts, of course garbage collection included, this job is very similar to a compiler, and is very complex. I also made our own mobile game using c++, I can show you the demo of client, if you like. I am very proficient in java also. I have a very good review on Freelancer.com, I never miss a project once I accept the job, you can check my review. Trust me, please let expert help you.
$300 USD in 7 days
4.9 (152 reviews)
7.0
7.0
User Avatar
Hi. I'm a programmer from Russia with strong skills in C++. You can read full list of my skills on my profile page. My plan is following. If you don't mind, I want to use Qt library. First, it provides instruments for network access, such as classes for interaction via TCP, HTTP or UDP protocols. Second, it provides classes for multithreading and temporary files. How I see the tasks: 1. First task. Module addresses some URL every 10 milliseconds to keep connection alive. It works in separate thread, so if connection freezes, it won't freeze the whole application. When block of data is read, it is cashed in temporary file. Module keeps the list of temporary files, so it will know the sequence of blocks. 2. Second task. We can keep list of alternative URLs in queue object. Each key is a latency for example. So the module will pick an alternative URL with lowest latency for example. But the concrete data structure is discussable. Another data structure is set (set of blacklist connections). If set contains the connection, it shouldn't be used. This operation has O(1) complexity. ----------------------------------------------- That's a few thoughts. It's late in Moscow and I'm sleepy, so we can discuss the project further tomorrow, when I'll have some sleep. If you're interested, feel free to contact me. Cheers, Sergey.
$500 USD in 20 days
4.9 (10 reviews)
5.3
5.3
User Avatar
Hi, I can do this. We are a team of 5 Experts in IoT, Embedded , Telecom, File System, Security & Scalable Server Professional in Linux, Unix, Windows, OSx, MacOS With C, Cpp, Shell Script, Python, Perl, django, XCode, Objective C, Swift OSS, Firmware, Device Driver, Linux System Calls, Board Bring up, Wireless network, Router Development, openWRT linux, Router tuning, OpenStack, IoT (Internet of Things) integration with Zigbee, Alljoyn, Home Automation using Raspberry-pi, Arduino , and Sensors. Have taught online courses in C and Shell Script. Domains: A. Embedded / IoT Domain B. Security Domain( DRM & Network Security) C. Telecom Domain( DNS/EDNS, VOIP, SIP), D. Embedded Domain( Virtualization & FileSystem Internals). E. Cloud(OpenStack) All with minimum 8 years of industry experience. Team have worked with QualComm, Cavium, Aricent, NEC, Juniper, LSI, Intel, Samsung for Development and Testing. -Rajesh
$1,052 USD in 10 days
5.0 (15 reviews)
4.6
4.6
User Avatar
Hi, I have 11 years of experience and understood your requirement. I work in distributed domain only. Your project is interesting and I like to work on it. Thanks, Anurag
$500 USD in 7 days
4.9 (17 reviews)
4.6
4.6
User Avatar
Hello, I understood the initial scope of this project. Although i want to discuss further this job in order to prepare the final concept for this project. After Complete discussion over the call or in chat, i will prepare following things for you - Technical Project Proposal - Flow chart for this Project - Execution plan (Step by step procedure with explanation how and at what that we are going to execute a particular task)
$515 USD in 18 days
5.0 (2 reviews)
4.3
4.3
User Avatar
Hi! I'm a system software engineer with the experience of working with plenty of Linux stuff on C including kernel modules. In order to complete this task I'll need to know the exact version of httpfs you're working with (including any possible 3rd-party patches) and the details of build environment. This job will definitely take some time for me, because the task is quite challenging. Looking forward to contact you. Denis.
$511 USD in 28 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of CZECH REPUBLIC
Sumperk, Czech Republic
5.0
84
Payment method verified
Member since Apr 5, 2009

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.