Task-conditional architecture offers advantage in parameter efficiency but falls short in performance compared to state-of-the-art multi-decoder methods. How to trade off performance and model parameters is an important and difficult problem. In this paper, we introduce a simple and lightweight task-conditional model called Prompt Guided Transformer (PGT) to optimize this challenge. Our approach designs a Prompt-conditioned Transformer block, which incorporates task-specific prompts in the self-attention mechanism to achieve global dependency modeling and parameter-efficient feature adaptation across multiple tasks. This block is integrated into both the shared encoder and decoder, enhancing the capture of intra- and inter-task features. Moreover, we design a lightweight decoder to further reduce parameter usage, which accounts for only 2.7% of the total model parameters. Extensive experiments on two multi-task dense prediction benchmarks, PASCAL-Context and NYUD-v2, demonstrate that our approach achieves state-of-the-art results among task-conditional methods while using fewer parameters, and maintains a significant balance between performance and parameter size.
20 March 2019 in Journal of Physics: Conference Series
Deep learning is a branch of the field of machine learning that outp-erforms humans in image and natural language processing. This mainly benefits from powerful computer processing capability and massive datasets. In the training model, the loss function needs to be used to evaluate each training result The quality of the loss function directly affects the correctness and validity of the model. In this paper, four kinds of common loss functions in deep learning are studied and our own loss function is proposed; Then the MNIST dataset is adopted to classify and train them; Finally, in the course of training, the change and the classification correct rate of the loss function value and model parameters are observed. Firstly, the different loss functions are experimented and their performance and application fields are analyzed, and secondly, our loss function is defined. After 30,000 times of iterative training, the test set data is used to test the loss function proposed, the correct rate reaches 98.53%, and achieves better accuracy in fewer training iterations, saving training times and resources.
16 April 2024 in Computer Vision and Image Understanding
There is an important class of applications in data centre called online dataintensive (OLDI) application, such as search engine, electronic shopping, and advertising, which has low latency, high throughput, and high burstiness. For better user experience, the query response delay in the data centre is controlled under 300 milliseconds. DCTCP can better meet the requirements of OLDI, but all the switches in the data centre are required to support RED. For data centre that does not fully support RED switch, TCP protocol can only be used. TCP Incast will appear in the application, which is far from the requirements of OLDI application. This paper proposes the TCP protocol of data centre based on window control. It is in existing switches and network devices,and centralized control makes the send window match with the switch queue cache. In the case of ensuring low latency, it keeps the switch queue in a small range of fluctuation, enabling the data centre to meet low latency, high throughput and high burstiness application requirements. Simulation experiments verify that WCTCP congestion control method can effectively solve the above problems. It is 5 times lower than TCPNewReno packet loss rate, 6 times higher than the network utilization rate, and the network performance is close to the DCTCP protocol.
Task-conditional architecture offers advantage in parameter efficiency but falls short in performance compared to state-of-the-art multi-decoder methods. How to trade off performance and model parameters is an important and difficult problem. In this paper, we introduce a simple and lightweight task-conditional model called Prompt Guided Transformer (PGT) to optimize this challenge. Our approach designs a Prompt-conditioned Transformer block, which incorporates task-specific prompts in the self-attention mechanism to achieve global dependency modeling and parameter-efficient feature adaptation across multiple tasks. This block is integrated into both the shared encoder and decoder, enhancing the capture of intra- and inter-task features. Moreover, we design a lightweight decoder to further reduce parameter usage, which accounts for only 2.7% of the total model parameters. Extensive experiments on two multi-task dense prediction benchmarks, PASCAL-Context and NYUD-v2, demonstrate that our approach achieves state-of-the-art results among task-conditional methods while using fewer parameters, and maintains a significant balance between performance and parameter size.